If you are looking to detect a multiplayer game, I would recommend reading the maxplayers cvar (except I think there is a better QuakeC way because what you really want is svs.maxclients exposed to QuakeC).
In a true single player game, maxplayers is 1.
However, I suspect reading the cvar is not the right way to do it in QuakeC .... I'll see if I can locate the correct method if someone doesn't beat me to it.
deathmatch and coop cvars have nothing to do with multiplayer, they have to do with game rules as QuakeC interprets them. For instance, start up Quake to the start map and type in the console "deathmatch 0; coop 0; map start". You'll find yourself sitting around on the start map. Type in the console "deathmatch 1; coop 1; map start" .... again you are sitting around on the start map. But you aren't in multiplayer.
Add:
Continued ... well, "maxplayers" is actually a command not a cvar so you cannot check the value ....