3.04bpw EXL3 quant of Ornith 1.0 397B. You'll need ~170+ GiB of VRAM to run it. Probably at least 7 x 3090 or 2x 6000 Pro. Use with TabbyAPI.
-- A perplexity: 3.20393975
-- B perplexity: 3.26597043
-- A label in top-K:
K = 1: 0.7186
K = 2: 0.8140
K = 3: 0.8549
K = 4: 0.8786
K = 5: 0.8937
-- B label in top-K:
K = 1: 0.7141
K = 2: 0.8116
K = 3: 0.8527
K = 4: 0.8767
K = 5: 0.8923
-- Top-K agreement, A vs B:
K = 1: 0.9422
K = 2: 0.7667
K = 3: 0.5596
K = 4: 0.3816
K = 5: 0.2486
-- KL divergence (A, B): 0.04249165
-- KL divergence (B, A): 0.03919449