We consider error estimates for some time stepping methods for solving fractional diffusion problems with nonsmooth data in both homogeneous and inhomogeneous cases. McLean and Mustapha \cite{mclmus} (Time-stepping error bounds for fractional diffusion problems with non-smooth initial data, Journal of Computational Physics, 293(2015), 201-217) established an $O(k)$ convergence rate for the piecewise constant discontinuous Galerkin method with nonsmooth initial data for the homogeneous problem when the linear operator $A$ is assumed to be self-adjoint, positive semidefinite and densely defined in a suitable Hilbert space, where $k$ denotes the time step size. In this paper, we approximate the Riemann-Liouville fractional derivative by Diethelm's method (or $L1$ scheme) and obtain the same time discretisation scheme as in McLean and Mustapha \cite{mclmus}. We first prove that this scheme has also convergence rate $O(k)$ with nonsmooth initial data for the homogeneous problem when $A$ is a closed, densely defined linear operator satisfying some certain resolvent estimates. We then introduce a new time discretization scheme for the homogeneous problem based on the convolution quadrature and prove that the convergence rate of this new scheme is $O(k^{1+ \alpha}), 0<\alpha <1 $ with the nonsmooth initial data. Using this new time discretization scheme for the homogeneous problem, we define a time stepping method for the inhomogeneous problem and prove that the convergence rate of this method is $O(k^{1+ \alpha}), 0<\alpha <1 $ with the nonsmooth data. Numerical examples are given to show that the numerical results are consistent with the theoretical results.