I reckon on a course like the Gorrick it's less than 1km/h slower - so approx 3 min or less on Sunday's race. But the difference is I was still quite fresh after 3 laps and could've gone for another, whereas with gears I would probably been close to collapsing.
At Beastway I could compare as I was riding both, I was always 1-1.5km/h slower than geared - but there was a lot of flat road sections not suited to singlespeeds.
In general I tend to prefer cranking up climbs standing up than spinning a small gear seated, so riding a ss suits me fine. Singletrack and technical sections are as fast if not faster as you always know in which gear you are and never do mistakes related to wrong gearing. There's only the downhills, and to an extent the flats, where you sometime have to resign to freewheel and lose a bit of time.
The psychological effect exists that's for sure