Where I Feel Like A Wimp And Look Like A Dork

Every Monday morning, since we only have one car and Big Man uses it to get to work, I take Carb Addict in a taxi to his weekly placement then come straight home again in the same cab. Since this started at the beginning of the year the fare has been, without fail, £11.60 but I always give the driver £12.50. Every driver I've had has gone a certain route. When we were in Liverpool last Monday Red took the trip and told me she was charged '£14-something'. I was annoyed because I thought they'd taken advantage of a young girl. Wrong.
Today we got in the taxi and the driver, an oldish man I'd not seen before started on a different route, saying that it was the same distance when I suggested the normal way. It's always busy at 9am because we live right by a school but once you're past that it's normally plain sailing however he used the traffic as a reason for his choice. I should've insisted that he went the normal way but reasoned with myself that if the fare ended up costing more than £11.60 I would refuse to pay and cite the regular price.
I don't need to tell you that the route the driver took entailed sitting in long traffic queues on the way back, do I? We were stationary and still around the corner from home when the meter hit £11.60.
'Let me out here' I said. The driver was a bit taken aback but pulled over and told me the cost, only for me to find that yet again, due to not wearing my glasses, I had put a 10p piece in my purse instead of a £2 coin. I only had £10.60. Feeling like the world's biggest gobshite I had to tell him that I needed to get more money from the house and that he would have to take me home after all. Which then meant we sat in a bit more traffic.
When we got to the house I must've looked at the meter and seen £12.80. I gave the driver the £10.60, ran into the house and got a £2 and a 20p and took it out to him. He sat looking at it puzzled. When I asked what was wrong he said that I'd only given him £2 and that it was short. I looked at the meter again and it was showing £13.20!
'It doesn't matter, love' the driver said, clearly feeling the waves of hostility radiating from me.
'Wait, I'll go and get it'
'No, really, it doesn't matter' he said.
'OK then. Bye' I said and stormed off.

Why didn't I insist on the driver taking the regular route? Or at least say that I always pay £11.60 and that I wouldn't pay any more? I've lived here 28 years and know as well as any cabbie where the traffic is bad. I didn't even get his number so I could ask NOT to be sent him next time, not that I think he'll be volunteering any time soon. The worst thing is looking such a bonehead when I'd tried to be clever and get out of the taxi before we got home.
Will I ever learn?