Rewrite chunk system (#8177)

Patch documentation to come

Issues with the old system that are fixed now:
- World generation does not scale with cpu cores effectively.
- Relies on the main thread for scheduling and maintaining chunk state, dropping chunk load/generate rates at lower tps.
- Unreliable prioritisation of chunk gen/load calls that block the main thread.
- Shutdown logic is utterly unreliable, as it has to wait for all chunks to unload - is it guaranteed that the chunk system is in a state on shutdown that it can reliably do this? Watchdog shutdown also typically failed due to thread checks, which is now resolved.
- Saving of data is not unified (i.e can save chunk data without saving entity data, poses problems for desync if shutdown is really abnormal.
- Entities are not loaded with chunks. This caused quite a bit of headache for Chunk#getEntities API, but now the new chunk system loads entities with chunks so that they are ready whenever the chunk loads in. Effectively brings the behavior back to 1.16 era, but still storing entities in their own separate regionfiles.

The above list is not complete. The patch documentation will complete it.

New chunk system hard relies on starlight and dataconverter, and most importantly the new concurrent utilities in ConcurrentUtil.

Some of the old async chunk i/o interface (i.e the old file io thread reroutes _some_ calls to the new file io thread) is kept for plugin compat reasons. It will be removed in the next major version of minecraft.

The old legacy chunk system patches have been moved to the removed folder in case we need them again.

1 files changed, 78 insertions, 0 deletions

diff --git a/patches/server/0206-Configurable-Alternative-LootPool-Luck-Formula.patch b/patches/server/0206-Configurable-Alternative-LootPool-Luck-Formula.patch
new file mode 100644
index 0000000000..40dce864c6
--- /dev/null
+++ b/patches/server/0206-Configurable-Alternative-LootPool-Luck-Formula.patch
@@ -0,0 +1,78 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Aikar <[email protected]>
+Date: Fri, 15 Jun 2018 00:30:32 -0400
+Subject: [PATCH] Configurable Alternative LootPool Luck Formula
+
+Rewrites the Vanilla luck application formula so that luck can be
+applied to items that do not have any quality defined.
+
+See: https://luckformula.emc.gs for data and details
+-----------
+
+The rough summary is:
+My goal was that in a pool, when luck was applied, the pool
+rebalances so the percentages for bigger items is
+lowered and smaller items is boosted.
+
+Do this by boosting and then reducing the weight value,
+so that larger numbers are penalized more than smaller numbers.
+resulting in a larger reduction of entries for more common
+items than the reduction on small weights,
+giving smaller weights more of a chance
+
+-----------
+
+This work kind of obsoletes quality, but quality would be useful
+for 2 items with same weight that you want luck to impact
+in varying directions.
+
+Fishing still falls into that as the weights are closer, so luck
+will invalidate junk more.
+
+This change will result in some major changes to fishing formulas.
+
+-----------
+
+I would love to see this change in Vanilla, so Mojang please pull :)
+
+diff --git a/src/main/java/net/minecraft/world/level/storage/loot/entries/LootPoolSingletonContainer.java b/src/main/java/net/minecraft/world/level/storage/loot/entries/LootPoolSingletonContainer.java
+index 710a66e9aafe8bd622f9f37789c281aba98d030e..d125b5a470be0f4c56c2c1f229fc5c83fc4d2f3b 100644
+--- a/src/main/java/net/minecraft/world/level/storage/loot/entries/LootPoolSingletonContainer.java
++++ b/src/main/java/net/minecraft/world/level/storage/loot/entries/LootPoolSingletonContainer.java
+@@ -113,9 +113,35 @@ public abstract class LootPoolSingletonContainer extends LootPoolEntryContainer
+     protected abstract class EntryBase implements LootPoolEntry {
+         @Override
+         public int getWeight(float luck) {
+-            return Math.max(Mth.floor((float)LootPoolSingletonContainer.this.weight + (float)LootPoolSingletonContainer.this.quality * luck), 0);
++            // Paper start - Offer an alternative loot formula to refactor how luck bonus applies
++            // SEE: https://luckformula.emc.gs for details and data
++            if (LootPoolSingletonContainer.this.lastLuck != null && LootPoolSingletonContainer.this.lastLuck == luck) {
++                return lastWeight;
++            }
++            // This is vanilla
++            float qualityModifer = (float) LootPoolSingletonContainer.this.quality * luck;
++            double baseWeight = (LootPoolSingletonContainer.this.weight + qualityModifer);
++            if (io.papermc.paper.configuration.GlobalConfiguration.get().misc.useAlternativeLuckFormula) {
++                // Random boost to avoid losing precision in the final int cast on return
++                final int weightBoost = 100;
++                baseWeight *= weightBoost;
++                // If we have vanilla 1, bump that down to 0 so nothing is is impacted
++                // vanilla 3 = 300, 200 basis = impact 2%
++                // =($B2*(($B2-100)/100/100))
++                double impacted = baseWeight * ((baseWeight - weightBoost) / weightBoost / 100);
++                // =($B$7/100)
++                float luckModifier = Math.min(100, luck * 10) / 100;
++                // =B2 - (C2 *($B$7/100))
++                baseWeight = Math.ceil(baseWeight - (impacted * luckModifier));
++            }
++            LootPoolSingletonContainer.this.lastLuck = luck;
++            LootPoolSingletonContainer.this.lastWeight = (int) Math.max(Math.floor(baseWeight), 0);
++            return lastWeight;
+         }
+     }
++    private Float lastLuck = null;
++    private int lastWeight = 0;
++    // Paper end
+ 
+     @FunctionalInterface
+     protected interface EntryConstructor {