They had cavalry divisions throughout the war. More at the end than at the start.
Their composition changed a lot though, and in the end they were used in so called cavalry-mechanized groups consisting of a cavalry corps (3 divisions each including a tank regiment of T-34s) and a tank or mechanized corps.
As an example the 1st Cavalry-Mechanized Group in July 1944, just before Operation Bagration that annihilated Army Group Center started, contained 133 T-34s, 47 T-60/T-70s, 50 Lend-Lease tanks, 20 SU-85, 49 SU-76, 1 ISU-152 and 3 SU-122.