String Safety

[revelator]

Sorry havent had time to test it yet but i suppose it should work also atleast it should give a hint as to where the buffer overflows.

[revelator]

this might explain it a bit better than i can.



look at the last comment.

besides the return on buffer overflow it actually works correctly but it makes it a pita to correct your code when the return simply kills your app,
or am i being to harsh here ?.

btw only trying to help here i dont intend to derail an otherwise interresting thread.

[revelator]

im wondering a bit if we could malloc the buffer dynamically ? that way it just grows to what ever size the buffer array needs.
though things like char string[somebuffersize]; would have to go but with a few checks we could probably ensure it newer grows larger than what the OS can handle hmm ?.
or would it chew to many resources ?.

btw the short answer to the question if _vsnprintf returns -1 is yes it triggers the crt buffer overflow check in msvcrt and kills the app but im not 100% it applies to the wrappers most of us use.
including the Q3 ones.

[Knightmare]

I induced an _vnsprinf overflow in KMQ2 (built under VS2008, I _vsnprinted the startup GL extension strings into a too-small buffer), and it didn't cause a crash. There were a few corrupt-looking chars at the end of the text, but KMQ2 didn't experience an error or crash.

The only place in KMQ2 where the return value from _vsnprint is checked is Com_sprintf, where it uses a 64KB temp buffer (far larger than the destination buffers it's copied into). So I don't think there's an actual cause for concern here.

[revelator]

its a bit strange... i get the crash when -1 returned on some versions of the msvcrt runtime but not all.
if it helps someone feel free to use the bsd version i posted but else dont since its so big.

[taniwha]

I might be coming into this a bit late, but QuakeForge has had "dynamic strings" (dstring) for a long time now (written by snax). Probably not that different (conceptually) from Sajt's stuff. They've almost completely replaced all occurrences of str*, and have even been pressed into service for things like building vertex arrays . Good old "va" is still heavily used (implemented using dstrings).

dsprintf, dasprintf, dvsprint and davsprintf ("a" appends the print output to the existing string).

String copying, inserting, appending, both null-terminated and "byte buffer", all supported. I doubt I've really used them to their full potential.

re _vnsprintf/vnsprintf: I've never experienced any problems with it in windows. I have, however, had a lot of fun with it in linux on various hardware (ppc etc), but VA_COPY and friends took care of that. However, looking at that msdn page, it looks like I should be passing size -1 rather than size. Easy fix.

That -1: just a while (vsnprintf (...) == -1) {grow buffer} No need for a special case for windows vs everybody else (actually, some other libc versions do -1 as well), though you do have to check whether you still need to grow the buffer if vsnprintf is implemented correctly and re-print, but it's still quite clean code. The only #if code in there is for VA_COPY.

[Baker]

[Spike]

try running FTE on a bigendian machine and you'll see if it supports it or not.
If I can't test it then I'm not going to pretend that it works.
Either way, the only mainstream system that really uses bigendian is old ppc macs, and those are slowly dying out.
I am ignoring PS3 consoles with the 'Other OS' option, but Sony also tried killing that too.
Its possible to get bigendian arm chips, but most are in littleendian mode.
Regarding FTE though, Bigfoot occasionally runs FTE on his MorphOS machines, which are bigendian ppc thingies. While its possible that certain extra features are not endian safe, the core featureset is or he would have complained about that by now.
For me, the easiest way to handle endian is to read/write one byte at a time, which is required for misaligned accesses (eg: arm) anyway, thus its silly to be calling LittleLong etc on incoming network data.
Ethernet through to UDP/TCP are all big-endian, yes, thus routers are often bigendian. (Net)Quake's transport layer is also bigendian, but the payload itself is little endian, if I remember correctly. Vanilla QuakeWorld will fail if not also fixed for bigendian (map hashes will kick you), but its protocol is fully little endian.

_vnsprintf on windows does not guarentee null termination. subtracting 1 is about as effective as just overwriting that byte afterwards too, which might give an easier compat path anyway. The return argument is more problematic, assuming you have something like quakeforge's dynamic strings.
Not sure msvc6 even has VA_COPY, but being x86 and fully stack based, its just a pointer or something trivial. I seem to remember having issues on more current chips, but I don't remember if it was arm or amd64 or both, either way VA_COPY is the proper way to fix that stuff (Baker: not an endian issue!).

[Baker]

I've been working with file formats lately and he said "ppc" so I drifted into off topic nonsequitur.

[taniwha]

I know of no x86 ABI that has VA_COPY (no need). ppc was the first one I encountered that needed it, then maybe mips (ps2), alpha and maybe arm. I wouldn't be surprised if pa-risc and i860 (? been too long. intel's crazy mixed-endian risc cpu) needed it too, but I wasn't doing much vsnprintf stuff then.

Every now and then, I get bug reports about endian issues with QF (solaris etc), but yeah, big-endian systems are rare these days. Continuing to support big-endian systems isn't a major problem for qf: the Little* functions just wind up as no-op assignments (a = LittleLong(a); -> a = a;). It's a problem only when I forget and somebody tries QF on something obscure.

Back to string safety: rather than worrying about getting strncpy/strncat (!!!) etc right every time (and worse, having arbitrary limits on strings), I prefer to use a nice dynamic string library: very difficult to mess up.

[Spike]

va_copy with no capitals is part of c99. 12+ years later and msvc still doesn't support va_copy.
ZQuake/FuhQuake/EZQuake/FTE all have a Q_strncpyz, which is basically strlcpy with no return value.
strlcpy/strlcat do just as you expect them to, rather than the annoying strncpy/bizzare strncat functions. Yay for BSD.
Its not infinite length, but it does the job.

[revelator]

BSD has rolled out some nice stuff indeed I use several of there functions on windows as replacements for not to optimal msvc implements.