Suggestions

A suggestion is text that the server streams into the editor: the answer of a language model to "Rewrite this", a summary, the next paragraph. The text shows as it comes, in a panel under the editable area, and goes into the document only when the person accepts it.

• The text is not in the document while it streams. A Markdown format cut in two by a chunk (**bo then ld**) is never half a format in the document.
• Accept inserts it as one undo step. Reject leaves the document and the selection as they were.
• The person can go on editing meanwhile: the suggestion goes where the selection was when it was asked for.
• A suggestion has only the formats and blocks of the editor's features.

The development server of this repository (mix dev) has an Assist menu answered by a fake model, with no API key.

Stream from the server

Four functions of Kotoba.Live push a stream to the editor with the id "post-body":

ref = Kotoba.Live.stream_ref()

socket
|> Kotoba.Live.stream_start("post-body", ref, at: :end, label: "Summary")
|> Kotoba.Live.stream_chunk("post-body", ref, "## Sum")
|> Kotoba.Live.stream_chunk("post-body", ref, "mary\n\nThe **main** point.")
|> Kotoba.Live.stream_end("post-body", ref)

In a real app the chunks come later, from a process (see A language model below). The functions:

• stream_start/4 opens the panel. A new stream in the same editor replaces the open suggestion.
• stream_chunk/4 adds text. The editor renders the whole text again at each chunk, so a chunk can end anywhere.
• stream_end/3 ends it: the person can now accept or reject it.
• stream_cancel/3 closes the panel with nothing inserted. Use it when the work fails: the live region says "The suggestion was cancelled".

ref names the stream. A stream that answers the Assist menu takes the ref of its request; one that the server starts by itself takes a new ref from Kotoba.Live.stream_ref/0. The editor ignores chunks of a ref that is not the open suggestion.

Where the text goes

:at chooses the place:

The selection is the one of the request (the Assist menu), or the editor's selection when the stream starts. When that text is no longer in the document (the person deleted it, or the server loaded a new document), the suggestion goes at the end.

Markdown or plain text

:format is :markdown (the default) or :text.

Markdown is read with the editor's own Markdown shortcuts: **bold**, _italic_, `code`, [links](https://example.com), ## headings, - lists, 1. lists, - [ ] check lists, > quotes, code blocks and ---. Markdown of a feature that the editor does not have stays text: in an editor without headings, ## Title is a paragraph with ## Title in it. A suggestion of a single paragraph at :selection or :caret goes into the paragraph, in the line; one with blocks adds blocks.

Plain text has no formats: a blank line starts a new paragraph, and a single newline is a line break.

The stored document is checked as always: Kotoba.Content sanitizes it, and Kotoba.Content.validate_features/3 checks its features (see Forms and changesets).

The Assist menu

Give the editor a list of actions with assist:

<.kotoba
  field={@form[:body]}
  id="post-body"
  assist={[rewrite: "Rewrite", summarize: "Summarize", continue: "Continue writing"]}
/>

The toolbar gets an Assist button, which opens a menu of the actions. An action is {id, label} (the id an atom or a string) or %{id: id, label: label}. assist={true} has no menu, only the events (for an extension's buttons, below); with no assist, the editor sends no assist events.

An action pushes kotoba:assist with a new ref, the action's id and the selected text ("" with no selection). Answer with a stream of that ref:

def handle_event("kotoba:assist", %{"ref" => ref, "action" => action, "text" => text}, socket) do
  socket = Kotoba.Live.stream_start(socket, "post-body", ref, at: at(action), label: label(action))
  {:noreply, start_writer(socket, ref, action, text)}
end

defp at("rewrite"), do: :selection
defp at(_action), do: :after

The editor then pushes kotoba:suggestion with the ref and an action:

• "accept": the person accepted it;
• "reject": the person rejected it, or the server loaded a new document over it, or started another stream;
• "stop": the person stopped it while it streamed, or the editor did (a new document, another stream, 30 seconds with no chunk, or the editor was removed from the page).

Stop the work on "stop" and "reject": the editor ignores the chunks that come after. What had come stays in the panel, and the person can still accept it.

Both events have the editor's id, and go to the editor's phx-target when it has one, as the other events do (see Forms and changesets).

From an extension

An extension can ask for a suggestion from its own button or shortcut with context.assist(action, detail). It pushes kotoba:assist as the menu does, with the keys of detail too, and returns the ref (or null when the editor has no assist):

register(editor, context) {
  return editor.registerCommand(TRANSLATE_COMMAND, (language) => {
    context.assist("translate", { language });
    return true;
  }, COMMAND_PRIORITY_EDITOR);
}

See the Extensions guide.

A language model

This LiveView streams the answer of Claude, from the Anthropic Messages API, with Req. The request runs in a task, which sends each text chunk to the LiveView. The LiveView keeps the task of each ref, to stop it when the person stops or rejects the suggestion.

Add a task supervisor to your application's children:

{Task.Supervisor, name: MyApp.TaskSupervisor}

The writer:

defmodule MyApp.Writer do
  @moduledoc "Streams Claude's answer to `lv` as `{:writer, ref, message}`."

  @url "https://api.anthropic.com/v1/messages"

  @system """
  You edit a document with its author. Answer with the new text only, in
  Markdown, with no preface and no comment.
  """

  def start(lv, ref, prompt) do
    Task.Supervisor.start_child(MyApp.TaskSupervisor, fn -> stream(lv, ref, prompt) end)
  end

  defp stream(lv, ref, prompt) do
    body = %{
      model: "claude-opus-5-5",
      max_tokens: 16_000,
      stream: true,
      system: @system,
      messages: [%{role: "user", content: prompt}]
    }

    result =
      Req.post(@url,
        json: body,
        headers: [
          {"x-api-key", System.fetch_env!("ANTHROPIC_API_KEY")},
          {"anthropic-version", "2023-06-01"}
        ],
        receive_timeout: 120_000,
        # A retry would send the chunks again.
        retry: false,
        into: fn {:data, data}, {req, resp} ->
          # Server-sent events: an event ends with a blank line, and a
          # piece of data can end in the middle of one.
          [rest | events] =
            (Req.Response.get_private(resp, :sse, "") <> data)
            |> String.split("\n\n")
            |> Enum.reverse()

          events |> Enum.reverse() |> Enum.each(&event(&1, lv, ref))
          {:cont, {req, Req.Response.put_private(resp, :sse, rest)}}
        end
      )

    case result do
      {:ok, %{status: 200}} -> send(lv, {:writer, ref, :done})
      _error -> send(lv, {:writer, ref, :error})
    end
  end

  defp event(event, lv, ref) do
    with "data: " <> json <- event |> String.split("\n") |> List.last(),
         {:ok, data} <- JSON.decode(json) do
      case data do
        %{"type" => "content_block_delta", "delta" => %{"type" => "text_delta", "text" => text}} ->
          send(lv, {:writer, ref, {:chunk, text}})

        # The model declined to answer, or the API failed in the stream.
        %{"type" => "message_delta", "delta" => %{"stop_reason" => "refusal"}} ->
          send(lv, {:writer, ref, :error})

        %{"type" => "error"} ->
          send(lv, {:writer, ref, :error})

        _other ->
          :ok
      end
    end
  end
end

The LiveView:

@editor "post-body"

def mount(_params, _session, socket) do
  {:ok, assign(socket, writers: %{})}
end

def handle_event("kotoba:assist", %{"ref" => ref, "action" => action, "text" => text}, socket) do
  {:ok, pid} = MyApp.Writer.start(self(), ref, prompt(action, text))

  socket =
    socket
    |> Kotoba.Live.stream_start(@editor, ref, at: at(action), label: label(action))
    |> update(:writers, &Map.put(&1, ref, pid))

  {:noreply, socket}
end

def handle_event("kotoba:suggestion", %{"ref" => ref}, socket) do
  {pid, writers} = Map.pop(socket.assigns.writers, ref)
  if pid, do: Task.Supervisor.terminate_child(MyApp.TaskSupervisor, pid)
  {:noreply, assign(socket, writers: writers)}
end

def handle_info({:writer, ref, message}, socket) do
  if Map.has_key?(socket.assigns.writers, ref) do
    {:noreply, writer(socket, ref, message)}
  else
    # A stopped or finished writer.
    {:noreply, socket}
  end
end

defp writer(socket, ref, {:chunk, text}), do: Kotoba.Live.stream_chunk(socket, @editor, ref, text)

defp writer(socket, ref, :done) do
  socket |> Kotoba.Live.stream_end(@editor, ref) |> update(:writers, &Map.delete(&1, ref))
end

defp writer(socket, ref, :error) do
  socket |> Kotoba.Live.stream_cancel(@editor, ref) |> update(:writers, &Map.delete(&1, ref))
end

defp prompt("rewrite", text), do: "Rewrite this text to make it clearer:\n\n#{text}"
defp prompt("summarize", text), do: "Summarize this text in one paragraph:\n\n#{text}"
defp prompt("continue", text), do: "Write the next paragraph after this text:\n\n#{text}"

defp at("rewrite"), do: :selection
defp at(_action), do: :after

defp label("rewrite"), do: "Rewrite"
defp label("summarize"), do: "Summary"
defp label("continue"), do: "Continue writing"

Some things to know:

• A task under Task.Supervisor is not linked to the LiveView: when the person leaves the page, it runs to its end, and its messages go nowhere. To stop it with the LiveView, stop the tasks of writers in terminate/2, or run it with Task.Supervisor.async_nolink/3 and handle its reply and :DOWN message.
• A long answer can take a minute. receive_timeout is the longest wait for one piece of data, not for the whole answer.
• The selected text goes to the model. Tell your users so, and do not send what they would not want sent.
• The model can write any Markdown. The editor keeps only its features, and the server checks the saved document, so the answer cannot add scripts, styles or links with other schemes (see Security).
• A chunk of text is a push to the browser. With a model that sends many small chunks, you can join them for 50 ms before a push; the editor renders at most once a frame anyway.

The keyboard and assistive technology

• The Assist button has aria-haspopup="menu" and aria-expanded. Enter, Space or a click opens the menu (role="menu", named "Assist"), with the focus on its first item. The Up and Down arrows, Home and End move between the items, Enter or Space asks for the suggestion, and Escape closes the menu, back to its button. After an item, the focus goes back to the editable area.
• The panel is a region named after the suggestion ("Rewrite"), with the text in it (aria-busy while it streams), a status ("Writing…", "Done", "Stopped") and its buttons: Stop while it streams, then Accept and Reject.
• In the editable area, Cmd/Ctrl+Enter accepts a suggestion that is done or stopped. Escape, in the editable area or in the panel, stops a suggestion that streams, and rejects one that is done or stopped.
• The live region says "Asked for a suggestion", "Rewrite: writing", "Suggestion ready: accept it with Ctrl+Enter, or reject it with Escape", then "Suggestion inserted", "Suggestion discarded" or "Suggestion stopped". It says "The suggestion was cancelled" for stream_cancel/3, and "The suggestion stopped" after 30 seconds with no chunk. It does not read the text as it streams: the person reads the panel when it is done.

Style

kotoba.css styles the panel (.kotoba-suggestion, with .kotoba-suggestion-title, .kotoba-suggestion-status, .kotoba-suggestion-content and the buttons .kotoba-suggestion-accept, .kotoba-suggestion-reject and .kotoba-suggestion-stop) and the menu (.kotoba-assist-menu, .kotoba-assist-item) with the --kotoba-* properties of the Theming guide.