Inappropriate behaviour and how to deal with it?

I was at the library a couple of days ago, sitting in a quiet area, reading a book and minding my own business. A guy came over and sat opposite me and I momentarily glanced up and then went back to my book. After a few minutes, I was aware that he was looking at me but I just continued reading and paid no heed. A few minutes more passed and the librarian came over and asked him a question. He didn’t respond, so she asked again. At this point I looked up as I thought it strange he hadn’t answered. He was still staring at me but eventually answered the question and I ignored him once again. I then became aware that his hand was moving and he began waggling his little finger to get my attention. When I looked, he had his penis out of his trousers, a full-on erection and he was wanking furiously grinning at me like an idiot. I was stunned by my own reaction. I was furious but the only thing that came out of my mouth was ‘do you fucking mind?’ He quickly zipped himself up and legged it pronto. I am in no way a prude and I think each to their own in privacy, but a public place like a library? It’s funny, but I was flashed a few years ago when out with a few mates and we found it hilarious and got great mileage out of it but I think it’s since I’ve had a child, incidents like this take on a whole new meaning. And that’s why I was so angry at the library, we were sitting about 40 feet away from the children’s section and any one of them could have strayed over and witnessed this.

I’m a little pissed off that I didn’t react more vociferously but I was stunned initially. How would you have reacted, seriously? Is there anything else I could/should have done?