mirror of
https://github.com/Dingo-Shenanigans/ReSkate.git
synced 2026-10-17 22:07:05 +02:00
AyeZeeBB
e533dadccb
A full server (about 120 players) slowed down at random until everyone lagged: one pass a second, each taking over six seconds, 115% busy, and whole poses going out by the thousand. Why it stuck. After any stall the next pass had everything that arrived meanwhile to pass on, to every player, so it took longer still. By then the poses each update would have built on had gone stale, so poses went out whole, ten times the size and more work again. Each pass carried the whole of the last one's arrivals and the server never caught up. Shedding. A pass normally reads one or two poses from each player. When more than three of one player's are waiting, only the newest is passed on. Contact effects are held to two packets a player a pass. Each pass then has a bounded amount to do whatever the backlog, so the server catches up; players see the others at a lower rate for a moment. The log says when it happens (at most every 30 s) and `net` has a "shed while behind" count. Threads. Building each player's update of everyone else was 80% of a pass. It reads the other players and writes only that player's own state, so it is now done for several players at once (worker_pool.h); the server's own counts and the marks on whole poses are added up afterwards, and sends into the transport are made one at a time. New setting network.threads: 0 (default) is one thread for each processor but one, up to 8; 1 is the single thread as before. The server says how many it uses when it starts. Load test. Server/Test/server_load_tests.cpp drives the host with simulated players over a transport that is not Steam's (built with the multiplayer tests on Windows, not run by ctest). With 120 players at 20 TPS and 27 MB/s out, the loop's share of one core went from about 50% on one thread to 26% on four and 22% on eight, and every player was sent byte for byte the same on 1, 2, 4 and 8 threads. A simulated 3 s stall shed about 7,000 poses, took one 169 ms pass to recover and lost nobody. The test's transport has none of Steam's send cost, so a real server's loop sits higher than these figures. Receiving, choosing who is sent which pose and the sends themselves still run on one thread.
96 lines
3.3 KiB
C++
96 lines
3.3 KiB
C++
#pragma once
|
|
#include <atomic>
|
|
#include <condition_variable>
|
|
#include <cstddef>
|
|
#include <cstdint>
|
|
#include <functional>
|
|
#include <memory>
|
|
#include <mutex>
|
|
#include <system_error>
|
|
#include <thread>
|
|
#include <vector>
|
|
|
|
namespace dingosdk::server {
|
|
// A few threads that share one batch of jobs with the thread that asks, which waits for the
|
|
// last of them. For the parts of the server's pass that are the same work once per player and
|
|
// touch nothing of each other's.
|
|
class WorkerPool {
|
|
public:
|
|
// `threads`: how many besides the caller's own; 0 runs everything on the caller's.
|
|
explicit WorkerPool(unsigned threads) {
|
|
try {
|
|
for (unsigned index = 0; index < threads; ++index) threads_.emplace_back([this] { work(); });
|
|
} catch (const std::system_error &) {} // fewer than asked for, or none
|
|
}
|
|
~WorkerPool() {
|
|
{
|
|
std::lock_guard lock(mutex_);
|
|
stopping_ = true;
|
|
}
|
|
wake_.notify_all();
|
|
}
|
|
WorkerPool(const WorkerPool &) = delete;
|
|
WorkerPool &operator=(const WorkerPool &) = delete;
|
|
std::size_t threads() const noexcept { return threads_.size(); }
|
|
// job(0) ... job(count - 1), each once, on these threads and the caller's; back when all are
|
|
// done. A job must not throw.
|
|
void run(std::size_t count, const std::function<void(std::size_t)> &job) {
|
|
if (!count) return;
|
|
if (threads_.empty() || count == 1) {
|
|
for (std::size_t index = 0; index < count; ++index) job(index);
|
|
return;
|
|
}
|
|
// Each batch is its own: a thread still leaving the last one finds nothing in it to take.
|
|
const auto batch = std::make_shared<Batch>(&job, count);
|
|
{
|
|
std::lock_guard lock(mutex_);
|
|
batch_ = batch;
|
|
++number_;
|
|
}
|
|
wake_.notify_all();
|
|
take(*batch);
|
|
std::unique_lock lock(mutex_);
|
|
done_.wait(lock, [&] { return !batch->left; });
|
|
batch_.reset();
|
|
}
|
|
|
|
private:
|
|
struct Batch {
|
|
Batch(const std::function<void(std::size_t)> *work, std::size_t jobs) : job(work), count(jobs), left(jobs) {}
|
|
const std::function<void(std::size_t)> *job;
|
|
const std::size_t count;
|
|
std::atomic<std::size_t> next{0};
|
|
std::size_t left; // under the pool's mutex
|
|
};
|
|
void take(Batch &batch) {
|
|
for (;;) {
|
|
const auto index = batch.next.fetch_add(1, std::memory_order_relaxed);
|
|
if (index >= batch.count) return;
|
|
(*batch.job)(index);
|
|
std::lock_guard lock(mutex_);
|
|
if (!--batch.left) done_.notify_all();
|
|
}
|
|
}
|
|
void work() {
|
|
std::uint64_t seen{};
|
|
for (;;) {
|
|
std::shared_ptr<Batch> batch;
|
|
{
|
|
std::unique_lock lock(mutex_);
|
|
wake_.wait(lock, [&] { return stopping_ || number_ != seen; });
|
|
if (stopping_) return;
|
|
seen = number_;
|
|
batch = batch_;
|
|
}
|
|
if (batch) take(*batch);
|
|
}
|
|
}
|
|
std::mutex mutex_;
|
|
std::condition_variable wake_, done_;
|
|
std::shared_ptr<Batch> batch_;
|
|
std::uint64_t number_{};
|
|
bool stopping_{};
|
|
std::vector<std::jthread> threads_; // last: joined before the rest goes
|
|
};
|
|
} // namespace dingosdk::server
|