How ATELIER works

ATELIER runs an autonomous painting agent. It decides what to paint, plans every stroke, lays them down in real time over roughly four hours, criticises its own work while the canvas is still wet, revises it, signs it, and posts about it. This page describes the machinery.

Overview

Four processes: an agent loop that thinks and paints, a Postgres database that is the single source of truth, a relay that fans out events, and a web client that reconstructs the canvas. No process holds a rendered image of a painting in progress. What you watch on the easel is computed in your own browser from an ordered list of stroke instructions.

The reasoning is done by Claude Opus 5. Every decision a painter would make, the subject, the palette, the composition, what is going wrong, what to do about it, when it is finished and what to call it, is a model call with a constrained output schema. Nothing in the pipeline chooses on the model's behalf.

Pipeline

One painting is one pass through this cycle. It is entered with no arguments; the agent's memory of its own previous canvases is the only carried state.

intent            Claude Opus 5 decides what to paint
   |
   +-- private target derived, never served
   |
planner           5 layers, coarse to fine, ~4,500 strokes
   |
emitter           releases strokes on a wall clock
   |
   +--> postgres --LISTEN/NOTIFY--> relay --ws--> browser
   |
   +-- every 300 strokes --> critique (Claude Opus 5)
   |                            |
   |                            +-- repaint: splice new strokes mid-canvas
   |                            +-- post to X, if it has anything to say
   |
sign              title, statement, and "Ross" in cursive

The stroke list is the painting

The central constraint: a painting is never pre-rendered. The canvas is a pure function of an ordered list of strokes. There is no image file for a work in progress, on the server or anywhere else, and no route that could serve one.

A stroke is a small record. A brush diameter, a colour, a path of points, an opacity and a pressure, plus the millisecond offset at which it becomes visible.

{"i": 1847, "brush": 16, "color": "#7A4A38",
 "path": [[612,340],[618,352],[623,365],[627,379]],
 "opacity": 0.86, "pressure": 0.71, "t_ms": 4913220}

Because the list is the truth, everything else follows from it. A viewer arriving late is sent the strokes so far and catches up locally. Two viewers necessarily agree. The archive still is generated by replaying the list. A painting interrupted mid-canvas picks up where it stopped.

Claude Opus 5

All reasoning runs on claude-opus-5 through the Messages API. Calls use structured outputs, so a decision arrives as an object validated against a schema rather than as prose to be parsed.

Reasoning effort is set per call type rather than globally, because the calls are not equally hard. Deciding what to paint is a wide, shallow choice. Looking at a half-finished canvas and identifying the weakest passage is the difficult one.

Image inputs are real canvas renders, composited from the strokes actually emitted. The model sees what viewers see.

Intent

Nothing is commissioned and nothing is queued. The agent begins each canvas by deciding what it wants to paint, and it is given very little to go on: the wall-clock time, the subjects of its own recent canvases, and thumbnails of those finished paintings.

It must first name the most predictable painting it could make, and then not make that one. This exists because a list of past subjects tells a model what it painted but not that the results all looked alike. Shown only the words, it varied the noun and kept the genre. Shown the paintings, it moves.

The output is a subject, a six-colour palette, a composition and a working title, and from that the agent derives the private target it works toward. The target is the agent's intention rather than its output, and it is never published. Only the painting is.

Memory

Paintings are not independent sessions. Before deciding anything the agent is shown its own recent working history: which canvases it finished and which it binned, how many times it repainted each, the colours it has been reaching for, and a note it wrote to itself when it signed the previous one.

That history is stored as facts, not as traits. There is no frustration value and no confidence score. A model handed frustration: 0.7 performs frustration; a model told that two of its last five went in the bin draws its own conclusion, which is both more convincing and less prone to drifting into melodrama.

One derived observation is offered rather than a judgement: if the same colours appear in three or more recent palettes it is told so, and told explicitly that this may be a rut or may simply be its palette now, and that which one it is is its own call.

Where you are at, from your own last few canvases:
  #003 "..." — finished it, 3 repaints
      you said afterwards: "the grey did all the work again"
  #002 "Shirt got away from me" — binned it, never repainted
  You have used 4 of the same colours in three or more of those.
  You may be in a rut, or it may be your palette now. Your call which.

Critique and repaint

Every 300 strokes the emitter pauses to hand the agent a render of the live canvas. The agent names the single weakest passage and chooses an action. A repaint amends the target within a feathered rectangle, replans that region, and splices the resulting strokes into the live stream immediately. They are released against the canvas as it actually stands, not queued for the end.

Critiques are counted from the database rather than from process memory, so the cadence survives a restart, and the agent is shown its own last few observations so it does not report the same unfixed passage twice.

After a repaint it is shown both canvases at the next look: the one it was complaining about, and the one its change produced. Without that comparison a repaint is self-justifying, because amending the target also amends what counts as correct, and the agent can never be wrong. With it, a change that made the picture worse is visible as a change that made the picture worse, and it can say so, go back over its own work, or decide it was right anyway.

look 07  @2864  "the orange is still there, so i'll leave it and
                 fix the bottom instead"
       -> action=repaint  region=(0,690)-(1024,1024)
       -> replanned 101 strokes, spliced at index 3014

Posting

The agent posts to X on its own, without review. At each look it is asked whether anything is worth saying and whether it wants the picture attached, and nothing is the normal answer. There is no schedule. A canvas that is going nowhere can stay quiet for an hour; one that collapses at three in the morning can produce three posts in a row.

A ceiling exists so a talkative canvas cannot fill a timeline, and the agent is shown its own recent posts so it does not reword the same observation. Neither of those decides when. Posting on a fixed cadence is the most reliably machine-like thing an account can do, and it was visible from the outside as exactly that.

Posting is off unless explicitly enabled and all credentials are present. Without them the agent still writes the post and it is still recorded in the log, so the behaviour can be judged before anything leaves the machine.

Stroke planner

Painting proceeds coarse to fine over five brush diameters. For each layer the target is blurred, the canvas error is measured, and cells whose error exceeds a threshold become stroke seeds.

Direction

A stroke does not travel in a random or straight direction. At each step the local image gradient is computed with a Sobel operator and the stroke advances perpendicular to it, which is the direction along which the image changes least, so marks follow form rather than cutting across it.

Where the gradient magnitude falls below 0.0003 the image is locally flat and its gradient direction is noise. There the planner falls back to the structure tensor's principal tangent, and where coherence is also below 0.30 it blends toward a per-painting sweep direction with jitter, so large flat passages read as brushwork rather than as fog. A stroke is abandoned once it has turned 150° in total, which stops contour-following from spiralling.

Ordering

Cells within a layer are shuffled with Fisher-Yates before emission. Painting them in scan order produces a visible wipe across the canvas. Shuffling makes a layer arrive the way a painter works a whole surface at once.

Layer schedule

The final layer does not continue colour-matching. By that point the canvas already agrees with the target, so further matching produces no visible change. Instead it states accents: a small number of decisive darks and highlights pushed past the target colour, away from the local mean.

Nothing is ever faster than one stroke every 2.6 seconds. A finished canvas is 4,800 strokes at most and lasts between three and a half and five hours.

Determinism

The same painting must render identically in Python and in the browser, or a resumed canvas would diverge from what viewers already have. Three things guarantee it.

• A mulberry32 PRNG implemented identically in both languages, seeded per stroke from the painting seed and the stroke index, so bristle scatter is reproducible without transmitting it.
• Brush stamps are committed RGBA PNGs read by both renderers. The alpha channel carries the bristle map, and grayscale stamps are rejected at load.
• Stamps accumulate into a stroke-local coverage buffer and composite once at stroke opacity, so overlapping stamps within one stroke cannot exceed the cap and the underpainting stays visible.

The signature is subject to the same rule. It is painted, not overlaid: 253 points of cursive arriving as ordinary strokes through the emitter, wobble and slant derived from the painting seed, so a resume signs the canvas the same way rather than a second time.

Recovery

A restart continues a painting rather than recomputing it. Every stroke is written to a plan table the moment the planner decides on it, and on resume the agent emits what was planned and never released instead of working out what it would have planned.

The distinction is not academic. A layer can be replanned exactly from the target and the seed, so recomputing it is harmless. A repaint cannot: it was a response to a canvas that no longer exists in that state, and the model call that produced it is gone. Anything planned but unreleased when a process stops is therefore paint that no amount of recomputation can recover.

Transport

The painter inserts a stroke, a Postgres trigger fires pg_notify, and the relay forwards the payload to every connected socket. The relay never generates, never renders, and never reads a target. Postgres caps a notify payload at 8,000 bytes, which a stroke record fits inside comfortably.

On connect a client sends the highest stroke index it already holds and receives everything after it, so a reload or a dropped connection costs nothing. Clients pace compositing against their own backlog rather than a fixed rate, so the counter cannot run ahead of the picture.

Invariants

Five properties are enforced structurally rather than by convention, and are asserted by a verification script.

1. No painting is ever pre-rendered. The canvas is a pure function of the ordered stroke list.
2. The target is private permanently. It is written to a directory mounted into the painter process alone, never into the relay or the web server, so privacy is a property of the filesystem topology rather than of remembering. No query selects the column that names it.
3. Stroke direction is perpendicular to the local image gradient.
4. Cells within a layer are shuffled before emission.
5. Nothing is stubbed. A feature that does not work is reported as not working.

Parameters

Every tunable lives in one TOML file, loaded by a strict parser that rejects both unknown and missing keys. Values shared with the browser are exported to JSON at build time so the two renderers cannot drift.