So anyway I asked him to mail me the bill instead so at least I would know I was sending a check to the right address, and he pressured me a bit not to do that. That made me even more suspicious. So I ended the call and just called Duquesne, fearing that I'd have to wait on hold for quite a while, but apparently nobody's awake at 9:30am and calling their electric company, and I got a real person like instantly. Turned out I did really owe the money, so I paid it by card after all.
I feel kind of annoyed at the guy for acting so suspicious since he was legit, but certainly to some extent the conversation got off to the wrong foot and then I started interpreting things as sketchy. I find it so annoying when someone calls me on a cell phone and is really reluctant to tell me where they're from or what they want --- I think he just said he was from an "professional business agency" in new york, and refused to tell me anything else until he confirmed I was in fact Jason Reed, though I suppose that's somewhat reasonable if he's working for a collections agency. Couldn't he at least have said "collections agency" off the bat, though?
Even then, being expected to give financial information to someone calling me without any information I can identify them by is really pretty sktechy.
This specific flavor of worrying that one is being social-engineered into giving away credit card numbers and such is also being fueled by the fact that I've been reading about phishing attacks that work by someone substituting unicode characters that resemble latin characters (like, say, a cyrillic lowercase a for a latin a) in, say, the domain paypal.com, registering the substituted domain, and getting you to somehow go to their site thinking it's the real paypal.