GraphIngest runs your workflows. An agent can watch them. Here is the path.

1. Connect the agent. Subscribe, create a key under Dashboard → API Keys, and add this listing in your editor. The agent can then see runs, read why one failed, restart it, and start a run.

2. Create a flow from steps we already have. Ask the agent to create a flow. It can use steps such as extract, then transform, then load. It saves that flow. It cannot open the folder on your computer, so a file you wrote is not included.

3. A step GraphIngest does not have, such as scrape_my_site. Write it on your computer. Set GRAPHINGEST_API_KEY to the GraphIngest API key on your subscription page, then call deploy(). That uploads the file. Asking the agent for that name only saves the name. The code goes up with deploy(). After the upload, the agent can watch that workflow.

Python:

import os
from graphingest import graph, deploy

os.environ["GRAPHINGEST_API_KEY"] = "paste the key from your subscription page"

@graph(name="scrape-my-site")
def pipeline(url: str):
    ...

deploy()

TypeScript:

import { graph, deploy } from "graphingest";

process.env.GRAPHINGEST_API_KEY = "paste the key from your subscription page";

const pipeline = graph({ name: "scrape-my-site" }, async (url: string) => {
  // your steps
});

await deploy();

Full product at www.graphingest.io.