Fabler

A multi-model AI story teller, built with Stable Diffusion, GPT, and neural text-to-speech (TTS).

Given a prompt as an opening line of a story, GPT writes the rest of the plot; Stable Diffusion draws an image for each sentence; a TTS model narrates each line, resulting in a fully animated video of a short story, replete with audio and visuals.

Quickstart

Check it out now or follow the instructions below to install locally.

Install from PyPi

$ pip install fabler

Install from Source

1. Clone the repository

$ git clone https://github.com/christopherwoodall/fabler.git

2. Install package requirements.

$ pip install --upgrade pip wheel
$ pip install -e ".[developer]"

3. Run the demo. The final video will be saved as /out/out.mp4, alongside other intermediate images, audio files, and subtitles.

$ fabler --scene=scene.yaml

4 Alternatively with make:

make install && make run

Usage

1. Load the model with defaults.

from fabler import Fabler

story_teller = Fabler.from_defaults()
story_teller.generate(...)

2. Alternatively, configure the model with custom settings.

from fabler import Fabler, FablerConfig

config = FablerConfig(
    writer="gpt2-large",
    painter="CompVis/stable-diffusion-v1-4",
    max_new_tokens=100,
    diffusion_prompt_prefix="Van Gogh style",
)

story_teller = Fabler(config)
story_teller.generate(...)

License

Released under the MIT License.