Swim propulsion strategies that work well in the macroscopic world turn out to be ineffective at the microscopic scale due to the dominance of viscous forces at very low Reynolds numbers. Understanding the locomotion strategies that can be adopted at the microscopic level by living microorganisms and synthetic microswimmers is of fundamental importance in biology and in biomedical applications. This work deals with the swimming of microscopic bodies in a highly viscous ambient fluid. We solve this fluid-solid interaction problem by means of a fully implicit formulation in which fluid unknowns (velocity and pressure) and positional degrees of freedom of the body are obtained simultaneously. This is accomplished by suitably constructing the space of kinematically admissible fluid-solid motions. A stabilized equal order formulation is adopted for the fluid part and linear triangular elements are used to approximate the swimmer geometry. Lagrangian update of the moving boundaries is performed either by a non-reversible Euler method or by a reversible mid-point scheme. In the latter case, a fixed-point iterative strategy is used to obtain the solution. We present a FEniCS-based finite element implementation of this fluid-swimmer interaction problem. Convergence of the proposed methodology is numerically assessed. Several numerical and implementation details are provided along challenging 2D- and 3D-axisymmetric examples, considering Newtonian and non-Newtonian rheologies.