Smitty,
First of all great post. As a boat owner I can share in your concerns and I would think most boat owners would agree in your assessments.
You hit the nail on the head when you stated there is no perfect boat, that being said one thing really stuck out to me.
Rivers.
I would take a drifter, nothing except a two man inflatable is going to be as comfortable to fish out, or maneuver. Get a high pedestal front seat and they are easy to cast from while seated.
On lakes the only draw back is the lack of speed. The way I view it a drifter can be very comfy on a lake, can be double anchored, three guys can fish, some can have a 9.9 long leg mounted to them.
Tough choice.