All Rav 4 Prime models can
All Rav 4 Prime models can shift into a manual mode by moving the shift lever to the left, the XSE just has paddles as an easier way to do it. When the engine is running in hybrid mode you can do simulated downshifts of the CVT to increase your engine RPM and power out of corners better. There also happens to be more deceleration when letting off the throttle due to higher engine rpm's.
When in EV mode you can also push the shifter to the left (or left/right paddles if you have them) and tap down/back to increase regen mode or up/forward to decrease regen mode. The effect is very obvious when you let off the throttle so don't take my word for it and try it yourself. Normal drive mode coasts with very little reduction in speed while the most aggressive regen mode in S1 will significantly reduce speed and show higher average regen which puts power back into your battery.
I've gotten 54 miles of range before my engine kicked in doing 3 days of drives to work plus a visit to the gym. Normally my commute consumes 8 EV miles each way but on 30-55mph roads the use of this feature reduces consumption to only 6 EV miles if I time traffic lights efficiently as I'm barely using the brakes.
As your car travels faster it will auto shift the regen mode to less aggressive modes again since the shifter & paddles can also increases engine RPM when in Hybrid mode. They probably didn't want you to shift the car to 9000 rpm when going 60+ mph if you toggled the EV/Hybrid button to turn the engine on.