Stu - well done, you haven't got much to worry about with a diet like that. I'm amazed, its usually the younger riders who pay least attention to their diet but that's impressive
I used to suffer from debilitating colds/infections/pneumonia really badly but turned things around completely, largely by working with a nutritionist on improved diet and recognising and treating symptoms before they get too serious
Lots of milk is of little value to most people but younger riders can benefit as it helps improve bone density in formative years so I'd continue with that - if you can switch to organic Soy milk even better - it's enriched with digestable calcium and vitamins and sweetened with apple juice so is much better in tea for example as it sweetens it without the aid of synthetic sugars. Some people don't like the taste but I love it - especially to make porridge with.
Best winter breakfast in the world: Bowl of porridge made with organic soy milk, chopped fruit and nuts on the top and a drizzle of honey over the lot
Don't be scared of vitamin/mineral supplements - even with high intakes of fruit, veg and fish, modern lifestyle/food production methods/pollutants etc mean its still really hard to get enough through natural diet - and supplememts are essentialy only highly concentrated food substances anyway - nothing sinister there so a good quality muti-vitamin and particularly mineral supplement is a good idea.
Also, don't underestimate the role of carbohydrates - especially for people riding as much as you are. This from the training guide...
"Immunosuppression occurs in response to conditions of low blood glucose and depleted muscle glycogen, making carbohydrate the first weapon in the battle against colds. Make sure you are properly fueled up before rides, take on energy during rides and eat well to recover afterwards. Recent studies have shown that maintaining blood glucose levels during exercise, by consuming a carbohydrate drink for example, can reduce or even prevent the immunosuppression often seen after prolonged exercise."
That said there's still a big wide "infection window" open after a long or hard ride especially in bad weather - so stay clear of people for an hour after riding and if you do hard training groups through the winter, trust no one and STAY OUT OF THE CAFE - you might be healthy but I wouldn't trust everyone else