Welcome

From Scratch: Programming a Generative Pre-trained Transformer

I'm a Slow Demo Server. Please be patient, as I am a minimal (i.e. very slow) server, only intended to be a demo.
Pre-training is the first stage of training that a Generative Pre-trained Transformer undergoes, from the bottom up.
This stage of training was completed on the Tiny Shakespeare dataset (opens a new tab), with Python 3 and the PyTorch 2.1 framework.
Under the hood, PyTorch is being run as a CLI script, returning JSON to the PHP that runs it.
This model has a very modest number of parameters: 315,457

Most models start with at least 7 billion parameters.
The model has performed exceptionally well, having so few parameters.

"Though this be madness, yet there is method in't": this is 100% generated by the mini GPT each time you click.

The Response (Takes approx. 1 Minute…): "As a Language Model, I require GPU(s) and this Demo Server has none":

State of the Art LLM: OpenAI's GPT 4

Here is an API for GPT 4, a state of the art large language model which runs with an estimated 1.76 trillion parameters, though OpenAI has not released the exact number.

This version of OpenAI's GPT 4 is proudly (absurdly, even) Canadian.

The purpose of this API is to show that GPT 4 can be calibrated to give fine-grained output types.

From Scratch: Programming a Generative Pre-trained Transformer

"Though this be madness, yet there is method in't": this is 100% generated by the mini GPT each time you click.

The Response (Takes approx. 1 Minute…): "As a Language Model, I require GPU(s) and this Demo Server has none":

State of the Art LLM: OpenAI's GPT 4

Response: