Diffusion MRI (dMRI) streamline tractography has been the gold standard for non-invasive estimation of white matter (WM) pathways in the human brain. Recent advancements in deep learning have enabled the generation of streamlines from T1-weighted (T1w) MRI, a more common imaging method. The accuracy of current T1w tracking methods is limited by their recurrent architecture. In the present work, we modify a current state-of-the-art T1w tractography method (CoRNN), replacing recurrent units and its sequential representation with Transformer modules, and modifying both the representation and the prediction network for the fiber orientation distributions. We demonstrate that these changes provide substantial performance benefits over the baseline method, producing high angular consistency with the gold standard dMRI tractogram in healthy normal adult humans.