To be precise, the websites don't have any say in that. Your DNS resolver resolves a domain name to its IP addresses and orders the results so that Unique Global IPv6 Addresses (UGAs) are preferred over Teredo over 6to4 over Unique Local Addresses (ULAs) over IPv4. The first IP of that sorting will be what the program causing the resolve will use to connect.
While it's true that IPv6 can get you a response faster over a pure IPv6 link, the fact you must tunnel v6 over v4 means you might even lose some throughput if trying to connect over v6 because inside the v4 packet the v6 packet will be encapsulated, limiting the effective payload size of a single packet (ergo, you need more packets for the same total payload size). Not to mention that, if the v4 link is crap, the v6 response will be crap, too, plus overhead.
In your case it'd be best to stick to v4 for now. Your only use case for v6 over v4 would be to reach v6-only websites, they're not exactly widespread.