Development and Deployment of Multiplayer Online Games, Vol. III.
Development and Deployment of Multiplayer Online Games, Vol. III.
Server Arch, Fault Tolerance, Pre-Coding
About the Book
What's the Big Idea?
The idea behind this book is to summarize the body of knowledge that already exists on multiplayer games but is not available in one single place. And quite a few of the issues discussed within this series (planned as
three nine volumes ~300 pages each), while known in the industry, have not been published at all (except for maybe in discussion forums).
Programming and deploying an MOG is a daunting task, and having a frame of reference is usually quite helpful — "hey, those guys have already tried this, and it worked for them."
Vol. I-III of the series are about MOG Architecture. In Vol. I and II, we discussed general issues (including Communication flows) and Client-Side Architecture respectively. By the end of Vol. III, we'll complete our discussion of the things related to architecting the Server-Side of your MOG, and this will conclude the ARCH part of the series. Vol. IV-VI will describe issues related to Programming, and Vol. VII-IX will be about Deployment and post-deployment tasks.
The book has got endorsements from quite a few people known in the industry, including people from Amaya, Lloyds Bank, Lionhead Studios, BlueByte (Ubisoft Studio), Lightstreamer, Plarium, and (formerly) Zynga. For the endorsements themselves please see Kickstarter campaign.
And speaking of crowdfunding - the book has already been funded on Kickstarter and Indiegogo - with over 800 backers total (and counting). BTW, here on Leanpub there is a bundle which is exactly the same as an ongoing InDemand campaign on Indiegogo - $20 for Vol. I-III together with an early access (though on Indie there is also an option to pre-order paperback or even hardcover).
Since When Are Rabbits Programming??
I cannot speak for all of rabbitkind, but this particular hare has gone alongside my writings for quite a while now. He was born as "No MT Bugs" Hare in 2010, in an Overload article titled "Single-Threading: Back to the Future?"
Therefore, there is evidence that this particular rabbit has been programming at least since 2010. Yes, he has changed his appearance, but deep inside, he is still the very same bunny with a naïve idea of making programs less buggy. BTW, the character is created by Sergey Gordeev, a guy with many animation awards, known (among other things) for being a director of some of the Animated Mr. Bean series.
Who Is this Book For?
The book is intended for those intermediary developers who want to become senior ones, and for senior developers and up. I expect that the book will also be useful for decision-makers, from PMs all the way up to CEOs — so if you by any chance know one, please make sure to give her a link to this page ;-).
On the other hand, if you are working on your first programming project, this book will probably be too difficult for you. There will be no explanation on what event-driven programming is all about, what's the difference is between optimistic locking and pessimistic locking, why you need a source control system, and so on. Instead, there will be discussions on how the concept of futures fits into event-driven programming, when the use of optimistic locking makes sense for games, and how to use source control in presence of unmergeable files.
In the same spirit, the book is not intended to be a CD to copy-paste your sample MOG from. Instead, the book aims to help you write your own code that is tailored to your own needs, and tries to advise on the available choices at most of the junctions.
What's The Plan?
Vol. III is currently in "2nd beta", with 80% of the content I'm planning, already included. Still, there will be more explanations.
As I'm working with professional editors (and vast majority of them doesn't know anything but Word :-( ) - I keep and edit my manuscript as a bunch of Word .docx files (I throw myself at the mercy of the court, and assure that it is only because of dire circumstances beyond my control ). I'm using Word to export .docx to PDF, and some_VBA_scripts+Calibre to export to .MOBI and .EPUB. However, .EPUB generated by Calibre, doesn't pass checks here on LeanPub (though it seems to show ok on iBook and on Clearview). If you want .EPUB - drop me a line, I will e-mail it to you.
Please note that current formatting is quite ugly; it will improve after being professionally formatted; it is also expected to pass all the .EPUB validity checks.
- The manuscript needs to be finalized; currently - Vol. III is about 80% text-complete.
- The manuscript will be sent to the "line editor" (that's what that Kickstarter money was for :-)).
- Then - it will be sent to the typesetter
- Then - it will be finalized here on Leanpub (and rewards for Kickstarter and Indiegogo backers will be sent out too)
- And finally, it will be published on Kindle (and on Amazon via CS). At this point, price of the book may go higher here on Leanpub (to match Kindle price).
All the comments are extremely welcome - and they DO help making the book better.
Chapter 8. Server-Side MOG Architecture
In-Memory States and Multi-Player Games
‘No Bugs’ Rule of Thumb for Multiplayer Games
Good for Us (as developers)
Exception: Stock Exchanges
NOT applicable to Single-Player Games
In-Memory Game World States: a Natural Fit for ‘No Bugs’ Rule of Thumb
On Data Consistency
In-Memory State Summary
On Scalability and Load Balancing
Scaling Up? No Way, Jose. Scaling Out!
Load Balancing of Stateful Objects. Two flavors of Load Balancing
Deployment Architectures, Take 1
Don't Do It: Naïve Game Deployment Architecture
Semi-Naïve Deployment Architecture
Web-Based Game Deployment Architecture
Web-Based Deployment Architecture (Web Stack): How It Works
Caching, More Caching, and Even More Caching
Taming DB Load: Write-Back Caches and In-Memory States
Write-Back Caches: When to Write Back?
Write-Back Caches: Locking
Caches and Transactions
Web-Based Deployment Architecture: Reactors
Web Deployment Architecture: Scaling and Load Balancing
On Specific Web Servers
Web-Based Deployment Architecture: Merits
Classical Game Deployment Architecture
Implementing Game Servers under Reactor-fest architecture
Reactors with External Interfaces
TCP Sockets Reactors and TCP Accept Reactor
Websocket-related Reactors and HTTP-related Reactors (not shown)
GPGPU Reactor (not shown)
Reactor-Fest Architecture on the Server Side: Flexibility and Deployment-Time/Run-Tim
Threads and Processes
Underlying Communication Protocol as an Implementation Detail
Moving Game Worlds Around (at the cost of client reconnect)
Online Upgrades with (Almost-)Zero Downtime
On Importance of Flexibility
Classical Deployment Architecture: Scaling and Load Balancing
Scaling Game World Servers - Natural Linear Scalability (except for seamless MMOs)
MMOGs and Seamless Game World Servers
Moving Objects Around
Scaling Matchmaking Server
Scaling Other Game Servers
Regional Datacenters to Reduce Latencies
Once again on hybrid Web-Based+Classical Architectures
Classical Game Deployment Architecture: Summary
Enter Front-End Servers
Front-End Servers as Concentrators
Front-End Servers: Benefits
Front-End Servers: Drawbacks and Issues to Solve
Latencies and Latency Differences
Discussion on Scalability of Front-End Servers and Dealing with “N-squared”. Server Gro
Front-End Servers: Implementation
Front-End Servers: Reactor-fest Implementation
Routing&Data Reactors in Game Servers and Clients
Client Load Balancing
Client-Side Random Balancing
Client-Side Random Balancing: a Law of Large Numbers, and comparison with DNS Round-R
Server-Side Load Balancer Appliances
Load Balancing Summary
Front-End Servers as a kinda-CDN
Front-End Servers + Game Servers as a kinda-CDN
Front-End Servers Summary
DB Server API
Meanwhile, at the King’s Castle… Implementing DB Server
Multi-Connection DB Server implemented as DB Reactor
Single-connection DB Server as DB Reactor
Implementing DB Server: Bottom Line
DB Server Summary
MOG Server-Side. Eternal Linux-vs-Windows Debate
New Generation Chooses Cross-Platform! Well, at least it SHOULD...
Eternal Windows-vs-Linux Debate
Other Technical Differences (scheduler, IPC, file access, etc)
Is it Enough to Decide?
Free as in “Free Beer”
On ISPs and Windows-vs-Linux Cost
Time To Market: Familiarity to your Developers
It is All about Money (
DB Server Considerations
Linux-vs-Windows: Time to Decide
Things to Keep in Mind: Windows
Things to Keep in Mind: Linux
Things to Keep in Mind: Reactors
MOG Server-Side. Programming Languages
Pros (compared to C++)
Cons (compared to C++)
Personal Preferences and Reactors
On Programming Languages as Such
Which Language is the Best? Or On Horses for Courses
On Programming Languages and Reactors
Supporting Multiple languages/compilers/JITs: Is It Worth the Trouble?
Supporting Different Programming Languages within the Same Project
Inter-Language Equivalence Testing: Yet Another Reactor Replay Benefit
Chapter 9. Fault Tolerance
Failure Modes & Effects
Game Server Failures
Containment of Game World server failures
Server Fault Tolerance: King is Dead, Long Live the King!
Clusters: HA, not FT
Fault-Tolerant Servers: Damn Expensive
Virtual Lockstep: Not Available Anymore?
Checkpoint-Based Fault Tolerance: Latencies and Even More Latencies (
DIY Fault-Tolerance in Reactor-fest architectures
DIY Virtual Lockstep
DIY N+1 Reactor-based Redundancy
DIY Fault Tolerance – Connections and IPs
DIY Fault Tolerance – What to Use
DIY Fault Tolerance in case of Almost-Determinism
DB Server Failures
It’s All about Numbers – and Nothing Else
Adding Fault Tolerance Can Make Your Numbers Worse
Exception – Stock Exchanges
/Implementing Fault Tolerance for DBs
Fault Tolerance – Failure Detection
Chapter 10. Pre-Coding Checklist: Things Everybody Hates, but Everybody Needs Them Too. From So
Git and unmergeable files
Git and 3rd-party Libraries
Protecting your Code
Reducing Chances: Firewalls & Antiviruses
Mitigating Impact from Compromises
3rd-party Libraries: Licensing
Issue Tracking System
Issue Tracking: No Bypassing Allowed
Enforcement and Static Analysis Tools
The Leanpub 60-day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms
80% Royalties. Earn $16 on a $20 book.
We pay 80% royalties. That's not a typo: you earn $16 on a $20 sale. If we sell 5000 non-refunded copies of your book or course for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earnedover $12 millionwriting, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.