ByteDance is pretraining a 10-trillion-parameter model
Sources say ByteDance's next foundation model is being trained at up to 10 trillion parameters — larger than any publicly disclosed training run and roughly an order of magnitude above the GPT-4 range. The model is expected to power future Doubao and enterprise AI products.