Deep learning method is of great importance in solving partial differential equations. In this paper, inspired by the failure-informed idea proposed by Gao et al. (SIAM Journal on Scientific Computing 45(4) (2023)) and as an improvement, a new accurate adaptive deep learning method is proposed for solving elliptic problems, including interface problems and convection-dominated problems. Based on the failure probability framework, the piece-wise uniform distribution is used to approximate the optimal proposal distribution and a kernel-based method is proposed for efficient sampling. Together with the improved Levenberg-Marquardt optimization method, the proposed adaptive deep learning method shows great potential in improving solution accuracy. Numerical tests on the elliptic problems without interface conditions, on one elliptic interface problem, and on the convection-dominated problems demonstrate the effectiveness of the proposed method, as it reduces the relative errors by a factor varying from $10^2$ to $10^4$ for different cases.