Retroquad protocol preview

[mankrip]

This is a preview of the protocol code I'm developing for Retroquad. It's the server side only, and the specifications aren't final, so I didn't bother to track and organize all details.

For an overview of the features being developed here, read the latest posts at my Tumblr blog.

This code is bigger and more complex than the original protocol code, so I don't think anyone else will want to implement it. But I'd like to know your impressions.

https://drive.google.com/file/d/0BxGzcs ... cslist_api

[Spike]

I was going to give a nice big reply, but the forum is still signing me out randomly and I took a while to actually bother picking everything apart.
so here's the highlights...

fitzquake's protocol sucks. copying and just changing the order of a few bits+bytes is stupid and pointless. compatibility goes out of the window for no good reason.
your protocol really doesn't offer anything new or novel, it just overcomplicates the code that uses it, making it a nightmare to figure out what is actually getting sent.

deltas from only baselines sucks and is just wasteful.
use nack-based deltas instead (LH refers to this as 'tribes networking' on account of the game that popularised it). Your server retains a copy of only the ONE last snapshot sent to the client, as well as a log of the bits that were changed. The client sends acks. If the server sees a gap in the acks then that gap is taken as a nack and the bits that were present in that frame are applied to the next outgoing snapshot ensuring the dropped fields are resent. The client is kept nice and simple.

SU_ stuff is stupid and absurd. just use stats instead. svc_updatestat doesn't need to be spammed. make a byte and float version of it if you so desire. this is what fte does. however, stuff that changes every frame (like weapon frame) requires that the data is flagged for resending as part of your network deltas (see above paragraph).
velocity should be part of the entity snapshot, and should be usable for determining the trajectory of things like nails, so you don't have issues with 16 players spamming their nailgun.

your baselines look like they're bordering on the bloated size. the signon buffer is likely still limited to 65k and thus if your baselines are larger than 6.5 bytes, then maps like helm18 (the 10k knights map) will not work. your entity bits alone are 8 bits.

absmin+absmax are absurd. use mins+maxs and significantly reduce the amount of data changing each frame.

you're not bounding your datatypes. an alpha of 1.01 will wrap to being very hard to see.

I don't see any code for 16bit angles nor larger coords. this is a deficiency in fitzquake that can be resolved with rmqe's protocol.

your update bits are badly ordered, resulting in excessive continuation bits and therefore bytes.
I assume you'll reorder once you get a greater idea for what it is that you're actually trying to do.

not saving globals simply because they are 0 breaks code like var foo = 1;

protocol-exclusive servers are deeply flawed. people are stuck either using a really crappy and limited protocol, or one that is so niche that it becomes irrelevant. a common trick is to send a 'cmd foo\n' stufftext and get the client to mis-respond somehow in a way that the server can detect without excessive abiguity.

you've completely misunderstood how items2 works, by the looks of it.

you have a load of weird and bizzare stuff like screenfov, and is just going to annoy people and conflict with DP_VIEWZOOM.

[mankrip]

Thanks for taking a look.

The datatypes are bounded when written to the message (see minmaxclamp). There's no reason to clamp them before, specially because I don't want to tamper with the internal state of the QC code.

16-bit angles/coords are implemented, I just didn't see a reason to include in the file since they're independent of the rest of the algorithm.

The way the vanilla protocol encodes items2 and serverflags is a nasty hack that can't be cleanly integrated with the rest, so I preferred to waste some bandwidth sending it separately.

I'm aware that DP has a zoom-related extension, but I didn't have the time to research into the DP code to avoid conflicts. That will be done later.

Weird and bizarre stuff is fun!

Yes, the bits will be reordered later.

I didn't understand the comment about var foo = 1; . That code for skipping empty values was implemented eons ago to reduce the savegames for the Dreamcast version of Makaqu, and it never caused any problem in any of the mods I've tried. I'll test this.

I've chosen abs min/max to avoid calculating it in the client, but you're right.

I haven't fully learned how the signon works, and I'm aware I'll have to study the buffers to make sure everything will fit. But thanks for the heads up.

Velocity isn't part of the entity snapshot in the vanilla protocol, which is why I haven't included it yet. But I'm planning to implement network prediction, so it will eventually be included.

The SU_ stuff was the last thing I've worked on, so it still needs some revisions. I've also thought "wtf" when I saw how it turned out…

I'm only using deltas from baselines because that's how the vanilla protocol works. I do want to implement nack-based deltas, but I didn't have time to study this yet.

About not offering anything new, well… I won't explain stuff just to argue.

[Baker]

You have "an idea" for a protocol.

Google search for the words "Spike" "protocol" "Quake" "InsideQC" and it will offer reading material.

You aren't going to get anything right the first time. I would humbly suggest coming back when you have a debugged implementation of a protocol.

Between now and then, you will learn 55 different things and make twice that many mistakes.

You will never make a network protocol without learning the 55 different things that "you don't know you don't know" and making several mistakes.

There are no shortcuts here.

[Spike may suggest to not even attempt this, i.e. no use for protocol only 1 engine can run. On the other hand, learning the guts of the protocols/networking is very helpful. It is also very time consuming.

And learning it by modifying a Quake engine may be the hardest and slowest way to learn it. Expect things you think will take a day or 2 to instead take a month if you use Quake as your playground.]

[mankrip]

The .alpha support in Makaqu 1.3 was pretty straightforward. When developing Makaqu 1.4, the code got incredibly more complex, much more than I had anticipated.

If I had posted the preliminary .alpha code from the 1.4 version, the reaction would have been the same. A pointless, stupid work.

But I had a reason to do things that way. I was aiming to make .alpha support on BSP entities possible, with proper overdraw between different entities but not between different surfaces of the same entity, and peacefully coexisting with the r_wateralpha, r_slime_blend_alpha, etc.

I didn't see anybody else manage to do the same. It was also a bit disappointing that nobody else managed to port that work to other engines.

Like then, I do have my own reasons to be refactoring the protocol code in this way. If I was just aiming to do what everyone else already did, what would be the point? Why develop my own engine then?

I'm aiming to do things that nobody else has done yet, but there's no reason for me to detail that, because I'm not asking for help. I just thought that maybe my code could have something useful or interesting to someone else.

If it doesn't, that's understandable. As I said in the first post, my expectations were low. But I decided to share anyway, just in case.

It was a mistake to share.

[Spike]

Please don't take this the wrong way - any distain I have for your protocol is the same as the distain I have for the fitzquake protocol. Neither solve the deficiencies with the vanilla protocol, they just expand upon it. For single player thats fine, but it results in bloated demos and laggy connections.

regarding items2, its a case of whether you put your hack in the server (where knowledge of existance of items2 and thus not poluting from serverflags isn't a problem), or in the client (where you really don't know which one to use meaning you have to use both simultaneously, which would be weird). Yes, either way its a hack.

Formula-izing the stuff that you send is a good idea, but using the same logic for everything all at once just makes your code unreadable. Yes baselines and deltas should share code for sending, but svc_clientdata really does need to (mostly) die, and trying to share code is basically futile because of that.
Being able to define certain fields, their allowed data ranges, etc, is a very nice way to control the set of data that is sent from ssqc->csqc for instance. However, in such cases you'll also need to track the entity 'class' so you know exactly which sets of fields are sent... as well as support csqc / equivelent.
I do aprechiate what you've tried to do with your code, but in my opinion you just took it to an extreme. If anything, your code just highlights exactly how many hacks and special cases the vanilla code had, by putting it all in a single unreadable location with all kinds of special cases and conditions instead of just doing the hack directly.

[Baker]

I think you misunderstand what I was saying. I was saying actively trying to make a perfect protocol or writing a new standard the first time is a mistake. It's too high a goal. Especially since you are trying to get social approval for a "theory", not a known working model.

DarkPlaces Protocol 7. I believe it was LordHavoc's 7th protocol attempt.

All I was saying: try to make a protocol that does what you need.

After you have that done, then worry about a perfect protocol that is documented and is some sort of standard.

Add: That + no one will take you seriously as a gardener if your boots aren't dirty.

[mankrip]

Thanks for the replies.

Anyway, I've put that work aside and went back to other parts of the engine for now.