From b3f6b759abcd40fd6e1e4ab699b9ecffe16c0b22 Mon Sep 17 00:00:00 2001 From: Justina Petraityte Date: Mon, 15 Jul 2024 15:54:57 +0100 Subject: [PATCH 1/3] adding guides for running ceramic-one --- .../composedb-server/running-locally.mdx | 6 +- docs/composedb/set-up-your-environment.mdx | 32 +- docs/introduction/technical-reqs.md | 16 +- .../guides/ceramic-nodes/running-cloud.md | 311 ++++++++++-------- .../guides/ceramic-nodes/running-locally.md | 66 +++- .../js-ceramic/networking/networking-index.md | 14 +- .../js-ceramic/networking/networks.md | 8 +- docs/protocol/js-ceramic/nodes/overview.md | 14 +- .../js-ceramic/nodes/running-a-node.md | 35 +- docs/protocol/js-ceramic/overview.md | 14 +- sidebars.ts | 24 -- src/components/homepage/tools-utilities.js | 2 +- static/img/protocol.png | Bin 0 -> 9549 bytes 13 files changed, 325 insertions(+), 217 deletions(-) create mode 100644 static/img/protocol.png diff --git a/docs/composedb/guides/composedb-server/running-locally.mdx b/docs/composedb/guides/composedb-server/running-locally.mdx index 81f0ea6c..890df36c 100644 --- a/docs/composedb/guides/composedb-server/running-locally.mdx +++ b/docs/composedb/guides/composedb-server/running-locally.mdx @@ -1,8 +1,10 @@ # Running Locally -Run a ComposeDB server on your local machine, e.g. your laptop +Run a ComposeDB server on your local server, e.g. your laptop. ## Things to Know -- ComposeDB requires running a Ceramic node (which uses IPFS) for decentralized data and a SQL instance for your index database. +- ComposeDB requires running a Ceramic node for decentralized data and a SQL instance for your index database. +- ComposeDB requires a running `ceramic-one` node which is responsible for storing the data and coordinating with network participants. + Make sure to configure and run the `ceramic-one` node first. You can find the steps of how to install and start the `ceramic-one` instance [here](../../set-up-your-environment#2-installation). - ComposeDB server can also be run locally [using Docker](../../guides/composedb-server/running-in-the-cloud.mdx). :::tip diff --git a/docs/composedb/set-up-your-environment.mdx b/docs/composedb/set-up-your-environment.mdx index a65be578..4a7ef9eb 100644 --- a/docs/composedb/set-up-your-environment.mdx +++ b/docs/composedb/set-up-your-environment.mdx @@ -21,6 +21,31 @@ There are a few ways to set up your environment. Choose the one that best fits y - [Using the Wheel](#installation-using-wheel) - the recommended and the easiest way to configure your full working environment and install the necessary dependencies. - [Using JavaScript package managers](#installation-using-javascript-package-managers) - an alternative, more manual, way to configure your working environment which supports `npm`, `pnpm` and `yarn`. +**Install and start the `ceramic-one` binary** + +All of the configuration options listed above **require a `ceramic-one` binary up and running**, which provides a data network access. You can run `ceramic-one` on your +local machine using two simple steps: + +1. Install the component using [Homebrew](https://brew.sh/): + +```bash +brew install ceramicnetwork/tap/ceramic-one +``` + +2. Start the `ceramic-one` using the following command: +```bash +ceramic-one daemon +``` + +:::note +By default, the command above will spin off a node which will connect to a `clay-testnet`. You can change this behaviour by providing a `--network` flag and specifying a network of your choice. For example: + +```ceramic-one daemon --network clay-testnet``` +::: + +With `ceramic-one` binary up and running you can move on with the ComposeDB installation and configuration method of your choice. + + --- ### 2a. Installation using create-ceramic-app @@ -145,8 +170,9 @@ GRANT ALL PRIVILEGES ON DATABASE "ceramic" to ceramic; ``` #### Configure the development environment +Make sure you have the `ceramic-one` binary up and running. To do that, follow the steps listed [here](#2-installation). -Once you have taken care of the prerequisites above, use Wheel to install all of the dependencies needed to run Ceramic and ComposeDB as well as configure the working environment +Now you can use Wheel to install all of the dependencies needed to run Ceramic and ComposeDB as well as configure the working environment for your project. To download Wheel, run the command below: @@ -296,6 +322,10 @@ node -v pnpm -v ``` +→ **ceramic-one** + +Make sure you have the `ceramic-one` binary up and running. To do that, follow the steps listed [here](#2-installation). + → **Ceramic** ComposeDB runs on Ceramic, so you will need to run a Ceramic node. To get started, we recommend running a local Ceramic node. If you're interested in running the production node, you can follow one of the [guides here](./guides/composedb-server/). diff --git a/docs/introduction/technical-reqs.md b/docs/introduction/technical-reqs.md index e2008819..6fe85116 100644 --- a/docs/introduction/technical-reqs.md +++ b/docs/introduction/technical-reqs.md @@ -9,13 +9,15 @@ Ceramic is a decentralized data storage network made up of different components, To make it easier to grasp, you can think about implementing Ceramic just like you might think about implementing a traditional SQL or PostgreSQL database. -When integrating with Ceramic, you will be running three pieces of server software: +When integrating with Ceramic, you will be running a few different services and components, each serving a specific purpose for running your application: -- Ceramic node -- PostgreSQL -- IPFS node +- `js-ceramic` - provides the HTTP API access for connected clients to read the streams stored on the Ceramic network +- `ceramic-one` - responsible for storing the actual data and coordinate with network participants. +- `PostgreSQL` - used for indexing data +- `Ethereum RPC node API access` - required to validate Ceramic Anchor Service (CAS) anchors. +- `Ceramic Anchor Service (CAS) access` - Anchors Ceramic protocol proofs to the blockchain. This service is currently funded by 3box Labs, however, eventually, this function will be provided by node operators and with some expected cost. -Ceramic and IPFS nodes are simply pieces of software than run on a server. PostgreSQL is a type of traditional database. +Ceramic nodes are simply pieces of software than run on a server. PostgreSQL is a type of traditional database. ## Hardware requirements @@ -46,13 +48,13 @@ As your project scales, you may need to expand your storage beyond 180GB. ### Advanced (heavy throughput) -Advanced users may want to consider running the PostgreSQL database on a different server than the Ceramic node + IPFS node. If you choose to run them on different servers, a VPC can be used to establish the communication between them. +Advanced users may want to consider running the PostgreSQL database on a different server than the Ceramic node. If you choose to run them on different servers, a VPC can be used to establish the communication between them.
-
Ceramic node + IPFS node
+
Ceramic node
diff --git a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md index 0aef49a8..b0a1d28a 100644 --- a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md +++ b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md @@ -1,26 +1,48 @@ -# Running Ceramic in production +# Running Ceramic nodes in the cloud environment --- -This guide provides complete instructions and various tools for launching a well-connected, production-ready Ceramic node. +This guide provides the instructions for launching a well-connected, production-ready Ceramic node in the cloud environment. ## Who should run a Ceramic node? --- -To run your application on `mainnet` you'll need to run your own production-ready node or to use a community provider like [hirenodes](https://hirenodes.io/). +To run your application on `mainnet` you'll need to run your own production-ready node or to use a community hosted nodes provider like [hirenodes](https://hirenodes.io/). ## Things to know --- -**Ceramic networks** – There are currently three Ceramic networks: `mainnet`, `testnet-clay`, and `dev-unstable`. Learn more about each network [here](../../networking/networks.md). By default, Ceramic will connect to `testnet-clay` and a [Ceramic Anchor Service](https://github.com/ceramicnetwork/ceramic-anchor-service) running on Gnosis. When you are ready to get on Ceramic `mainnet`, check out [this guide](../../../../composedb/guides/composedb-server/running-in-the-cloud) to get access to our `mainnet` anchor service. +**Ceramic networks** +There are currently three main Ceramic networks: +- `mainnet` +- `testnet-clay` +- `dev-unstable` -**Running IPFS** – Ceramic relies on a system called [IPFS](https://docs.ipfs.io/) to connect to and share data in Ceramic networks. IPFS runs as a separate process from the Ceramic node itself, with each Ceramic node connected to a dedicated IPFS node over HTTP. The Ceramic Daemon can launch an IPFS process automatically (referred to as running IPFS in "bundled" mode in the Ceramic config file), which is designed for testing and local development only. For production deployments you should run your own IPFS process manually and point your Ceramic node at it (referred to as running ipfs in "remote" mode in the Ceramic config file). This allows for more configuration options for your IPFS node allowing for more controlled resource allocation, as well as improved maintenance, debugging and observability. Note that Ceramic only supports `go-ipfs` version 0.12 or later. +Learn more about each network [here](../../networking/networks.md). -The rest of this guide assumes you are running Ceramic with IPFS in "remote" mode. +By default, Ceramic will connect to `testnet-clay` and a [Ceramic Anchor Service](https://github.com/ceramicnetwork/ceramic-anchor-service) running on Gnosis. When you are ready to get on Ceramic `mainnet`, check out [this guide](../../../../composedb/guides/composedb-server/access-mainnet) to get access to our `mainnet` anchor service running on Ethereum mainnet. -**Process management, restarts and data persistence** – Ceramic and IPFS will not automatically restart if they crash. You should configure your own restart mechanism, and you must ensure data persistence between restarts. +**Supported platforms** – You can run Ceramic nodes on a cloud provider of your choice. This guide will include instructions for the Digital Ocean Kubernetes, but the +instructions can be applied to the vast majority of other cloud providers like AWS and others. + +**Supported Operating Systems:** + +- Linux +- Mac +- Windows + +:::note +For Windows, Windows Subsystem for Linux 2 (WSL2) is strongly recommended. Using the Windows command line is not portable and can cause compatibility issue when running the same configuration on a different operating system (e.g. in a Linux-based cloud deployment). +::: + +**Compute requirements:** + +You’ll need sufficient compute resources to power `ceramic-one`, 'js-ceramic' and `PostgreSQL`. Below are the recommended requirements: + +- 4 vCPUs +- 8GB RAM ## Required steps @@ -28,201 +50,218 @@ The rest of this guide assumes you are running Ceramic with IPFS in "remote" mod Below are the steps required for running a Ceramic node in production. This guide will teach you how to: -1. [Install and run the Ceramic daemon](#running-the-daemon) -2. [Configure data persistence](#data-persistence) -3. [Get connected to the network](#get-connected-to-the-network) -4. [Get observability data from your node (optional)](#observability) -## Quick start +### Configure your Kubernetes Cluster ---- +Running a Ceramic Node on DO Kubernetes will require two tools: -### [Run Ceramic on AWS ECS with Terraform →](https://github.com/ceramicnetwork/terraform-aws-ceramic) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) - the Kubernetes command line tool +- [doctl](https://docs.digitalocean.com/reference/doctl/how-to/install/) - the Digital Ocean command line tool -The 3Box Labs team has written a [Terraform module](https://github.com/ceramicnetwork/terraform-aws-ceramic) that configures Ceramic and IPFS in AWS ECS using Fargate. Using this module is a fast and reliable way to run Ceramic in the cloud because it is set up for data persistence and auto-restarts. The module currently requires some common AWS resources to be pre-configured as well as Cloudflare. See an [example of the module in use](https://github.com/ceramicnetwork/terraform-aws-ceramic/blob/main/examples/ecs/main.tf). +Make sure you have these tools installed on your machine before proceeding to the next step of this guide. -We highly encourage the community to create Terraform modules or other templates for different infrastructure providers to further decentralize the Ceramic network. +To create a Digital Ocean Kuberetes cluster, follow an official [DigitalOcean tutorial](https://docs.digitalocean.com/products/kubernetes/how-to/create-clusters/). The process of setting up your Kubernetes cluster will take about 10 minutes. Once it’s up and running, you are good to continue with the next step. -## Running the daemon +### Connect to your Kubernetes Cluster ---- +Once the cluster is up and running, you will be provided a command that you can use to authenticate your cluster on your local machine. You will be provided with a command unique to your cluster, but For example: -The [js-ceramic](https://github.com/ceramicnetwork/js-ceramic) node is run as a daemon using Node.js or Docker. +```doctl kubernetes cluster kubeconfig save 362dda8b-b555-4c47-9bf0-1a81cf58e0a8``` -By default, the Ceramic daemon runs bundled with a [go-ipfs](https://github.com/ipfs/go-ipfs) node and connects to the Clay testnet and Gnosis [Ceramic Anchor Service](https://github.com/ceramicnetwork/ceramic-anchor-service). In production, you should change these defaults to secure your data and accommodate your infrastructure setup. +Run this command on your local machine using your local terminal. After authenticating, verify the connectivity: -The Ceramic daemon can be configured with a JSON file which is created on start and located at `$HOME/.ceramic/daemon.config.json` by default (you can also point to a custom location for the config file using the `--config` flag when starting the Ceramic Daemon). See [example daemon.config.json](#example-daemonconfigjson) below. +```kubectl config get-contexts``` -### Run with Docker containers +### Deploy Ceramic -Docker images to run Ceramic and IPFS are built from the source code of the [js-ceramic](https://github.com/ceramicnetwork/js-ceramic) and [go-ipfs-daemon](https://github.com/ceramicnetwork/go-ipfs-daemon) repositories respectively. Images built from the main branches are tagged with `latest` and the git commit hash from which the image was built. You can view the image builds of [js-ceramic on DockerHub](https://hub.docker.com/r/ceramicnetwork/js-ceramic). The Docker image for `go-ipfs-daemon` pre-configures IPFS with plugins that make it easy to run on cloud infrastructure. You can view the image builds for [go-ipfs-daemon on DockerHub](https://hub.docker.com/r/ceramicnetwork/go-ipfs-daemon). +Running a Ceramic node will require configuring three components: +- `ceramic-one` - a binary which contains the Ceramic Recon protocol implementation in Rust +- `js-ceramic` - component which provides the API interface for Ceramic applications +- `postgres` - a database used for indexing -### Run outside of containers +To simplify the configuration of all these services, you can use the [SimpleDeploy](https://github.com/ceramicstudio/simpledeploy/tree/main), a set of infra scripts that will make the configuration process faster and easier. -If you would like to run Ceramic and IPFS outside of containers or on bare metal, start by installing [go-ipfs](https://github.com/ipfs/go-ipfs) (**version 0.12 or later**). Depending on your infrastructure setup you may consider building `go-ipfs` with the [healthcheck plugin](https://github.com/ceramicnetwork/go-ipfs-healthcheck) and [S3 datastore plugin](https://github.com/3box/go-ds-s3). Once IPFS is installed, configure it to use pubsub, which Ceramic relies on for message passing. This can be done by running the IPFS daemon with the `--enable-pubsub-experiment` flag or modifying the configuration by running `ipfs config --json Pubsub.Enabled true` (learn more in the [IPFS docs](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-pubsub)). Next you can run IPFS and install the Ceramic daemon with the [js-ceramic CLI](https://www.npmjs.com/package/@ceramicnetwork/cli), which is available as a public NPM module. It is currently compatible with **Node.js versions 14 and 16.** +1. Clone the [simpledeploy](https://github.com/ceramicstudio/simpledeploy.git) repository and enter the created directory: -## Data Persistence +``` +git clone https://github.com/ceramicstudio/simpledeploy.git +cd simpledeploy +``` -To run a Ceramic node in production, it is critical to persist the [Ceramic state store](#ceramic-state-store) and the [IPFS datastore](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#datastorespec). The form of storage you choose should also be configured for disaster recovery with data redundancy, and some form of snapshotting and/or backups. +2. Create a namespace for the nodes: -**With ComposeDB** +``` +export CERAMIC_NAMESPACE=ceramic-one-0-17-0 +kubectl create namespace ${CERAMIC_NAMESPACE} +``` -Additionally, if using ComposeDB, your persistence strategy also needs to account for backups/snapshotting of your Postgres instance. In this case, your backup process should implement the following order: +3. Create ephemereal secrets for js-ceramic and postgres -1. Snapshot your Postgres instance first -2. State store -3. IPFS block store +``` +./scripts/create-secrets.sh +``` -Leveraging this order guarantees that the higher-level subsystems won't know about data that the lower-level subsystems are missing in the backup. +4. Apply manifests: -**Loss of this data can result in permanent loss of Ceramic streams and will cause your node to be in a corrupt state.** +``` +kubectl apply -k . +``` -The Ceramic state store and IPFS datastore are stored on your machine's filesystem by default. The Ceramic state store defaults to `$HOME/.ceramic/statestore`. The IPFS datastore defaults to `ipfs/blocks` located wherever you run IPFS. +5. Wait for the pods to stat. It will take a few minutes for the deployment to pull the docker images and start the containers. You can watch the process with the following command: -The fastest way to ensure data persistence is by mounting a persistent volume to your instances and configuring the Ceramic and IPFS nodes to write to the mount location. The mounted volume should be configured such that the data persists if the instance shuts down. +``` +kubectl get pods --watch --namespace ceramic-one-0-17-0 +``` -#### IPFS Datastore +You will know that your deployment is up and running when all of the processes have a status `Running` as follows: -The IPFS datastore stores the raw IPFS blocks that make up Ceramic streams. To prevent data corruption, use environment variables written to your profile file, or otherwise injected into your environment on start so that the datastore location does not change between reboots. +```bash +NAME READY STATUS RESTARTS AGE +js-ceramic-0 0/1 Running 0 77s +rust-ceramic-0 1/1 Running 0 77s +postgres-0 1/1 Running 0 77s +``` -Note: Switching between data storage locations is an advanced feature and should be avoided. Depending on the sharding implementation you may need to do a data migration first. See [https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#datastorespec](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#datastorespec) for more information. +Hit `^C` on your keyboard to exit this view. -#### Ceramic State Store +:::note -The Ceramic state store holds state for pinned streams and the acts as a cache for the Ceramic streams that your node creates or loads. To ensure that the data you create with your Ceramic node does not get lost you must pin streams you care about and you must ensure that the state store does not get deleted. +You can easily access the logs of each of the containers by using the command below and configuring the container name. For example, to access the Ceramic node logs, you can run: +`kubectl logs --follow --namespace ceramic-one-0-17-0 js-ceramic-0` -## Examples +::: ---- +### Connecting to Ceramic -### Example with Docker containers +The Ceramic daemon serves an HTTP API that clients use to interact with your Ceramic node. The default API port is `7007`. Make sure this port is available to all clients you plan to use for your application. -```bash -docker pull ceramicnetwork/go-ipfs-daemon:latest +:::caution -# Use this snippet to keep the datastore in the volume -docker run \ - -p 5001:5001 \ # API port - -p 8011:8011 \ # Healthcheck port - -v /path_on_volume_for_ipfs_repo:/data/ipfs \ - --name ipfs \ - go-ipfs-daemon + Healthchecks can be run against the API endpoint `/api/v0/node/healthcheck`. +::: -# Get the IP address -docker inspect -f \ - '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \ - ipfs -``` -Before running the Ceramic daemon, configure it to use IPFS in "remote" mode. See [example daemon.config.json](#example-daemonconfigjson) below. +### Connect to the mainnnet anchor service +By default, your Ceramic node will connect to the Ceramic `clay-testnet`. In order to connect your application to the mainnet, you will have to configure your node and verify you node DID for using the Ceramic Anchor Service (CAS). You can find a detailed step-by-step guide [here](../../../../composedb/guides/composedb-server/access-mainnet). -```bash -docker pull ceramicnetwork/js-ceramic:latest -# Use this snippet to keep the statestore in a volume -docker run -d \ - -p 7007:7007 \ - -v /path_on_volume_for_daemon_config:/root/.ceramic/daemon.config.json \ - -v /path_on_volume_for_ceramic_logs:/root/.ceramic/logs \ - -v /path_on_volume_for_ceramic_statestore:/root/.ceramic/statestore \ - -e NODE_ENV=production \ - --name js-ceramic \ - ceramicnetwork/js-ceramic:latest -### Example without containers +--- -After installation, both daemons can be run from the command line +### Example with Docker containers -```bash -ipfs init -ipfs daemon -``` +All state in this configuration is ephemeral, for persistence use docker-compose. -Before running the Ceramic daemon, configure it to use IPFS in "remote" mode. See [example daemon.config.json](#example-daemonconfigjson) below. +1. Start rust-ceramic using the host network -```bash -ceramic daemon +```json +docker run --network=host \ + public.ecr.aws/r5b3e0r5/3box/ceramic-one:latest ``` -### Example daemon.config.json +2. Start js-ceramic using the host network ```json -{ - "anchor": { - "ethereum-rpc-url": "https://eg_infura_endpoint" // Replace with an Ethereum RPC endpoint to avoid rate limiting - }, - "http-api": { - "cors-allowed-origins": [ - ".*" - ] - }, - "ipfs": { - "mode": "remote", - "host": "http://ipfs_ip_address:5001" - }, - "logger": { - "log-level": 2, // 0 is most verbose - "log-to-files": true - }, - "network": { - "name": "mainnet", // Connect to mainnet, testnet-clay, or dev-unstable - }, - "node": {}, - "state-store": { - "mode": "fs", - "local-directory": "/path_for_ceramic_statestore", // Defaults to $HOME/.ceramic/statestore - } -} +docker run --network=host ceramicnetwork/js-ceramic:develop ``` -## Get connected to the network +### Docker-compose ---- - -### Connecting to Ceramic - -The Ceramic daemon serves an HTTP API that clients use to interact with your Ceramic node. The default API port is `7007`. Make sure this port is available to all clients you plan to use for your application. - -:::caution +1. Create a testing directory, and enter it. - Healthchecks can be run against the API endpoint `/api/v0/node/healthcheck`. -::: +```yaml +mkdir ceramic-recon +cd ceramic-recon +``` -### Staying connected to IPFS +2. Create a file colled `docker-compose.yaml` with the configuration shown in the example below and save it: -Ceramic nodes rely on IPFS for networking. IPFS nodes connect to each other using a Libp2p module called "switch" (aka "swarm"). This module operates over a websocket, on port `4011` by default. The websocket port must be accessible to the internet so your Ceramic node can be connected to the network. +``` +version: '3.8' + +services: + ceramic-one: + image: public.ecr.aws/r5b3e0r5/3box/ceramic-one:0.19.0 + network_mode: "host" + volumes: + - ceramic-one-data:/root/.ceramic-one + + js-ceramic: + image: ceramicnetwork/js-ceramic:develop + environment: + - CERAMIC_RECON_MODE=true + network_mode: "host" + volumes: + - js-ceramic-data:/root/.ceramic + - ./daemon.config.json:/root/.ceramic/daemon.config.json + command: --ipfs-api http://localhost:5001 + +volumes: + ceramic-one-data: + driver: local + js-ceramic-data: + driver: local +``` -:::caution +3. Update the js-ceramic configuration file `daemon.config.json` with the configurations provided below. - Healthchecks can be run against the `HEALTHCHECK_PORT` (port `8011` by default) when `HEALTHCHECK_ENABLED` is `true`. +:::note +The js-ceramic configuration file can be found using the following path: `$HOME/.ceramic/daemon.config.json ` ::: -Additionally, when running IPFS the IPFS API port must be accessible by the Ceramic node. The default API port is `5001`. The IPFS node address will then be passed to Ceramic with the `ipfs.host` option in the Ceramic daemon config file. - -### Connect to the mainnnet anchor service -For nodes that wish to connect to Ceramic mainnet, the node's IP address will have to be added to the allowlist for the Ceramic Anchor Service node operated by 3BoxLabs. Once you have fully configured your Ceramic node with this guide and have a way to persist its configuration and state, open an issue in the [Ceramic Anchor Allowlist Repo](https://github.com/3box/ceramic-anchor-allowlist) with the public, static *egress* IP address for your Ceramic node, and a brief description of the data persistence setup for the multiaddress, Ceramic State Store, and IPFS Repo. Once your issue is closed, you will be connected to the Ceramic network and the [Ceramic Anchor Service](https://github.com/ceramicnetwork/ceramic-anchor-service). +```json +{ + "anchor": { + "auth-method": "did" + }, + "http-api": { + "cors-allowed-origins": [ + ".*" + ], + "admin-dids": [ + ] + }, + "ipfs": { + "mode": "remote", + "host": "http://localhost:5001" + }, + "logger": { + "log-level": 2, + "log-to-files": false + }, + "metrics": { + "metrics-exporter-enabled": false, + "prometheus-exporter-enabled": true, + "prometheus-exporter-port": 9465 + }, + "network": { + "name": "testnet-clay" + }, + "node": { }, + "state-store": { + "mode": "fs", + "local-directory": "/root/.ceramic/statestore/" + }, + "indexing": { + "db": "sqlite://root/.ceramic/db.sqlite3", + "allow-queries-before-historical-sync": true, + "disable-composedb": false, + "enable-historical-sync": false + } +} +``` -Nodes that wish to connect to other Ceramic networks, such as the clay testnet, do not need to do anything special to gain access to the Ceramic Anchor Service. The Anchor Service for the `testnet-clay` network is open to everyone and does not have an IP allowlist like the mainnet service does. +3. Run `docker-compose up -d` -:::caution - Mainnet nodes will not run immediately after start up until your pull request is reviewed and your IP address is added to the allow list for the 3Box Labs hosted anchor service. -::: +--- -## Observability ---- -Ceramic has a debug mode that you can enable using the `--debug` flag. This will allow you to see all logs printed to your console, including debug logs, API requests, events, and errors. -For observability, it is best to have these logs written to files to debug any issues and to generate metrics. Logging to files can be enabled with the `logger.log-to-files` config file option. The default location for logs is `~/.ceramic/logs` but this path can be configured with the `logger.log-directory` config file option. Even without debug mode enabled you will still get critical logs and metrics written to files. -Request and event logs are written in [logfmt](https://brandur.org/logfmt). This makes them easy to import into [Grafana](https://grafana.com/) dashboards using a log scraping agent like [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) and a log aggregator like [Loki](https://grafana.com/docs/loki/latest/), which can be used as a data source for Grafana. An example of such a setup can be found [here](https://github.com/3box/ceramic-stats). -## Next steps ---- -Congratulations! You have now set up a well-connected Ceramic node in the cloud which can receive HTTP requests from the local environment, the [JS HTTP Client](../ceramic-clients/javascript-clients/ceramic-http.md), or to simply serve as another node to replicate and pin streams. Please report any bugs as issues on the [JS Ceramic GitHub](https://github.com/ceramicnetwork/js-ceramic). \ No newline at end of file diff --git a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md index 218c0312..acc5e812 100644 --- a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md +++ b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md @@ -2,26 +2,51 @@ --- -The Ceramic command line interface provides an easy way to start a JS Ceramic node in a local Node.js environment. This is a great way to get started developing with Ceramic before moving to a cloud-hosted node for production use cases. +To run a local Ceramic node you will generally need to run two key components: +- `js-ceramic` - an api interface for Ceramic applications +- `ceramic-one` - a binary that provides a Ceramic data network access through the protocol implementation in Rust. + +You should always start with running the `ceramic-one` component first to make sure that the `js-ceramic` component can connect to it. ## Prerequisites --- -Installing the CLI requires a console, [Node.js](https://nodejs.org/en/) v16, and [npm](https://www.npmjs.com/get-npm) v6. Make sure to have these installed on your machine. +Installing the `js-ceramic` requires the following: +- a terminal of your choice, +- [Node.js](https://nodejs.org/en/) v20, +- [npm](https://www.npmjs.com/get-npm) v10 + +Make sure to have these installed on your machine. + + +## Setting up the `ceramic-one` component + +The easiest way to install the `ceramic-one` is using [Homebrew](https://brew.sh/) package manager. After installing Homebrew on your local machine, you can install `ceramic-one` using the following command: -:::caution +```bash +brew install ceramicnetwork/tap/ceramic-one +``` - While npm v7 is not officially supported, you may still be able to get it to work. You will need to install the `node-pre-gyp` package globally. This is required until `node-webrtc` which IPFS depends on [is upgraded](https://github.com/node-webrtc/node-webrtc/pull/694). +Once installed, run the ceramic-one binary by running the command provided below. Not that using the flag `--network` you can modify the network: +```bash +ceramic-one daemon --network testnet-clay +``` - npm install -g node-pre-gyp +:::note +There are many flags for the daemon CLI that can be passed directly or set as environment variables. You can pass the `-h` flag to see the complete list as follows: +```ceramic-one daemon -h``` ::: -## Installation and usage +You also have an option of running the `ceramic-one` binary using Docker. Check out the instructions in the [README of rust-ceramic repository](https://github.com/ceramicnetwork/rust-ceramic?tab=readme-ov-file). + + +## Setting up the `js-ceramic` component + +The Ceramic command line interface provides an easy way to start a JS Ceramic node in a local Node.js environment. This is a great way to get started developing with Ceramic before moving to a cloud-hosted node for production use cases. ---- ### Install the Ceramic CLI @@ -31,22 +56,41 @@ Open your console and install the CLI using npm: npm install -g @ceramicnetwork/cli ``` -### Launch the Ceramic node +### Launch the `js-ceramic` node -Use the `ceramic daemon` command to start a local JS Ceramic node connected to the [Clay Testnet](../../networking/networks.md#clay-testnet) at `https://localhost:7007`. +Use the `ceramic daemon` command to start a local JS Ceramic node connected to the [Clay Testnet](../../networking/networks.md#clay-testnet) by default running at `https://localhost:7007`: ```bash ceramic daemon ``` -### **Configure your network** +### Configure your network (Optional) By default, the JS CLI starts a node on the [Clay Testnet](../../networking/networks.md#clay-testnet). If you would like to use a different network, you can specify this using the `--network` option. View [available networks](../../networking/networks.md). Note, the CLI can not be used with [Mainnet](../../networking/networks.md#mainnet). -### **Configure a node URL** +### Configure a node URL (Optional) It is possible to use the CLI with a remote Ceramic node over HTTP, instead of a local node. To do this, use the `config set` command to set the `ceramicHost` variable to the URL of the node you wish to use. ```bash ceramic config set ceramicHost 'https://yourceramicnode.com' ``` + +## Monitoring +You can always check if `js-ceramic` and `rust-ceramic` components are available by running the commands listed below. + +### `js-ceramic` service's availability + +Check the `js-ceramic` service’s availability with the healthcheck endpoint: + +```json +curl http://localhost:7007/api/v0/node/healthcheck +``` + +### `rust-ceramic` service's availability + +Check the rust-ceramic service’s availability with the liveness endpoint: + +```json +curl http://127.0.0.1:5001/ceramic/liveness +``` \ No newline at end of file diff --git a/docs/protocol/js-ceramic/networking/networking-index.md b/docs/protocol/js-ceramic/networking/networking-index.md index d94a3d3e..710e47e3 100644 --- a/docs/protocol/js-ceramic/networking/networking-index.md +++ b/docs/protocol/js-ceramic/networking/networking-index.md @@ -1,6 +1,6 @@ # Networking -Networking sub-protocols for Ceramic +Networking sub-protocols for Ceramic. ### Overview @@ -13,15 +13,3 @@ Networks are collections of Ceramic [nodes](../nodes/overview.md) that share spe ### [Data Feed API](data-feed-api.md) The Ceramic Data Feed API gives developers a way to keep track of all the new state changes that are happening in the Ceramic network. This enables developers to customize the way their data is indexed and queried, and enables the development of new custom database products on top of Ceramic. - -### [Tip Gossip](tip-gossip.md) - -When a stream is updated, the latest event (tip) is gossiped and propagated out to all the nodes in a network that are interested in that particular stream. Listening for all tips, allows a node to learn about streams it did not know about. - -### [Tip Queries](tip-queries.md) - -Nodes in a network with a specific StreamId can query for the most recent event (tip) of that given stream. Queries enable a node that know about a stream to find the latest event (tip). - -### [Event Fetching](event-fetching.md) - -Nodes that have the tip (latest event) of a stream, can use the tip to fetch all prior events in that stream. Fetching enables a node that knows a tip to sync the entire event set for a stream and learn its latest state. \ No newline at end of file diff --git a/docs/protocol/js-ceramic/networking/networks.md b/docs/protocol/js-ceramic/networking/networks.md index 3e6dd814..5acb8159 100644 --- a/docs/protocol/js-ceramic/networking/networks.md +++ b/docs/protocol/js-ceramic/networking/networks.md @@ -5,9 +5,9 @@ Information about the default Ceramic networks ## Overview --- -Networks are collections of Ceramic [nodes](../nodes/overview.md) that share specific configurations and communicate over dedicated [libp2p](https://libp2p.io/) pubsub topics. Networks are disjoint from one another; streamsthat exist on one network are **not** discoverable or usable on another. +Networks are collections of Ceramic [nodes](../nodes/overview.md) that share specific configurations and communicate over dedicated [libp2p](https://libp2p.io/) pubsub topics. Networks are disjoint from one another; streams that exist on one network are **not** discoverable or usable on another. -These pubsub topics are used to relay all messages for the defined networking sub protocols, including [Tip Gossip](tip-gossip.md) and [Tip Queries](tip-queries.md). +These pubsub topics are used to relay all messages for the defined networking sub protocols. ## All Networks --- @@ -19,11 +19,11 @@ An overview of the various Ceramic networks available today: | Mainnet | mainnet | /ceramic/mainnet | Ethereum Mainnet (EIP155:1) | Public | | Clay Testnet | testnet-clay | /ceramic/testnet-clay | Ethereum Gnosis Chain | Public | | Dev Unstable | dev-unstable | /ceramic/dev-unstable | Ethereum Goerli Testnet | Public | -| Local | local | /ceramic/local-$(randomNumber) | Ethereum by Truffle Ganache | Private | | In-memory | inmemory | | None | Private | -!!!note +:::note There is currently a proposal to decompose each network into multiple pubsub topics for scalability, the pubsub topics will remain prefixed by the network identifier `/ceramic//` see [CIP-120](https://github.com/ceramicnetwork/CIP/blob/main/CIPs/cip-120.md) +::: ## Public networks --- diff --git a/docs/protocol/js-ceramic/nodes/overview.md b/docs/protocol/js-ceramic/nodes/overview.md index 4bc1e2fe..d9654cd9 100644 --- a/docs/protocol/js-ceramic/nodes/overview.md +++ b/docs/protocol/js-ceramic/nodes/overview.md @@ -1,26 +1,14 @@ # Nodes Overview --- -Two types of nodes currently work together to support the Ceramic network. ## Ceramic Nodes ---- - A Ceramic node is a bundle of services and long-lived processes that support the protocol and provide access to the Ceramic Network. Current implementations bundle and run most all of the following services and sub protocols defined here. This includes the following: -### IPFS Services - -The following services are typically provided to a Ceramic node by a connected IPFS node. These services do not necessarily have to be provided through an IPFS node or all bundled together. - -| IPLD Blockstore | Stores the underlying IPLD blocks for event streams. Supports block formatting and content addressing. | -| --- | --- | -| Gossipsub | Several layers of the libp2p stack are used including GossipSub to query streams and broadcast stream updates in the network. | -| Kademlia DHT | A distributed hash table for content and peer lookup and discovery. The same DHT table is used as the primary IPFS network. | -| Bitswap | Exchange and sync blocks with peers. Allows an event stream to be sycned from one node to another. | ### **Ceramic Services** -| Pinning | Tracks the streams a node wants to store and to receive the latest events for. | +| Service | Description | | --- | --- | | StateStore | Tracks and stores the latest tips for pinned streams and caches stream state. | | Networking | Runs the stream query and update protocols on Gossipsub and manages peer connections. | diff --git a/docs/protocol/js-ceramic/nodes/running-a-node.md b/docs/protocol/js-ceramic/nodes/running-a-node.md index f885c821..45a76d46 100644 --- a/docs/protocol/js-ceramic/nodes/running-a-node.md +++ b/docs/protocol/js-ceramic/nodes/running-a-node.md @@ -13,6 +13,11 @@ This can be installed from NPM and updated through NPM by using the following co npx @ceramicnetwork/cli daemon ``` +:::note +Make sure that you have `ceramic-one` binary running in the background. To set it up, follow the setup steps [here](../guides/ceramic-nodes/running-locally#setting-up-the-ceramic-one-component). +::: + + This will install the CLI and start the daemon. This will allow all of the initial files to be created. This will successfully have a node running on the Clay TestNet. ## Operations Considerations @@ -55,9 +60,31 @@ sudo systemctl start logrotate ### Monitoring -It is strongly recommended to use your existing monitoring system to collect and process the [metrics offered by the node](../../../composedb/guides/composedb-server/server-configurations.mdx). In addition to those, it is strongly recommended to monitor and avoid exhaustion of: +It is strongly recommended to use your existing monitoring system to collect and process the [metrics offered by the node](../../../composedb/guides/composedb-server/server-configurations.mdx). + + +#### Availability + +Check the `js-ceramic` service’s availability with the healthcheck endpoint + +```json +curl http://localhost:7007/api/v0/node/healthcheck +``` + +Check the `rust-ceramic` service’s availability with the liveness endpoint + +```json +curl http://127.0.0.1:5001/ceramic/liveness +``` + +#### Metrics -* the amount of free space available on all relevant filesystems -* the amount of free [inodes](https://en.wikipedia.org/wiki/Inode) on all filesystems containing IPFS files +Both `rust-ceramic` and `js-ceramic` have prometheus compatible endpoints available. + +`rust-ceramic` is enabled by default + +```jsx +curl http://127.0.0.1:9464/metrics # rust-ceramic metrics +``` -These can be checked by hand using `df -h` for space and `df -i` for inodes. \ No newline at end of file +js-ceramic monitoring configuration is described [here](https://developers.ceramic.network/docs/composedb/guides/composedb-server/server-configurations#prometheus-endpoint0). \ No newline at end of file diff --git a/docs/protocol/js-ceramic/overview.md b/docs/protocol/js-ceramic/overview.md index d6d80a75..fca3b8c2 100644 --- a/docs/protocol/js-ceramic/overview.md +++ b/docs/protocol/js-ceramic/overview.md @@ -2,5 +2,17 @@ Ceramic is a decentralized event streaming protocol that enables developers to build decentralized databases, distributed compute pipelines, and authenticated data feeds, etc. Ceramic nodes can subscribe to subsets of streams forgoing the need of a global network state. This makes Ceramic an eventually consistent system (as opposed to strongly consistent like L1 blockchains), enabling web scale applications to be built reliably. -The protocol doesn't prescribe how to interpret events found within streams; this is left to the applications consuming the streams. One example of this type of application is [ComposeDB](../../composedb/getting-started). + +The latest release of Ceramic has introduced a new Rust-based implementation of Ceramic protocol which offers performance and stability improvements as well as a new data synchronisation protocol called Recon. Developers, building on Ceramic network will be using two main components: +- `js-ceramic` component which provides the API interface for Ceramic applications +- `rust-ceramic` component which provides Ceramic data network access (contains the implementation of Recon protocol). + +
+![protocol-overview](/img/protocol.png) +
+ + +The protocol doesn't prescribe how to interpret events found within streams; this is left to the applications consuming the streams. Some examples of this type of application are: +- [OrbisDB](https://useorbis.com/) +- [ComposeDB](../../composedb/getting-started) diff --git a/sidebars.ts b/sidebars.ts index 33c9493c..fa299d2a 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -94,11 +94,6 @@ const sidebars: SidebarsConfig = { type: "doc", id: "protocol/js-ceramic/guides/ceramic-clients/javascript-clients/queries", label: "Queries" - }, - { - type: "doc", - id: "protocol/js-ceramic/guides/ceramic-clients/javascript-clients/pinning", - label: "Pinning" } ] }, @@ -123,18 +118,6 @@ const sidebars: SidebarsConfig = { label: "DID Session" } ] - }, - { - type: "category", - collapsed: true, - label: "Stream APIs", - items: [ - { - type: "doc", - id: "protocol/js-ceramic/guides/ceramic-clients/stream-api/caip10-link", - label: "Using CAIP-10 Links" - } - ] } ] } @@ -188,13 +171,6 @@ const sidebars: SidebarsConfig = { items: [ { type: "doc", id: "protocol/js-ceramic/networking/networks", label: "Networks" }, {type: "doc", id: "protocol/js-ceramic/networking/data-feed-api", label: "Data Feed API" }, - { type: "doc", id: "protocol/js-ceramic/networking/tip-gossip", label: "Tip Gossip" }, - { type: "doc", id: "protocol/js-ceramic/networking/tip-queries", label: "Tip Queries" }, - { - type: "doc", - id: "protocol/js-ceramic/networking/event-fetching", - label: "Event Fetching" - }, ] }, { diff --git a/src/components/homepage/tools-utilities.js b/src/components/homepage/tools-utilities.js index a1b269f3..8d360a5e 100644 --- a/src/components/homepage/tools-utilities.js +++ b/src/components/homepage/tools-utilities.js @@ -8,7 +8,7 @@ const FeatureList = [ items: [ { url: "docs/protocol/js-ceramic/overview", - text: "Dive into the JavaScript implementation of the Ceramic Protocol." + text: "Dive into the specifications and implementation of the Ceramic Protocol." } ] }, diff --git a/static/img/protocol.png b/static/img/protocol.png new file mode 100644 index 0000000000000000000000000000000000000000..0d355de08b853e2eb151c31e9aa622b65d608938 GIT binary patch literal 9549 zcmY*<1z1#F*ES^#N(`VV3`jRfcc&mFARr(h&Cp194N?v*9ZGi&Ig|qe($X<>iL`Y8 z!{ht>-}}yW&73o5_E~%HHEZ8%o%`Mquhm}SKO}#MhK7c(q$sC}hK6B}I)4YiMtzFa z1ZSfT=q{QsWzZ^yD7H{vJG_3n3s5d$4jfRH#1sx3&bwt0v z_66hLUoq^zVE%iK_U^vnY&haA8XA3rlAQD#PxRf)N65$8WZi@Z@|reS=9bbnIiC%G zv;c!6^B=H)(=3A%^%IL}-Wu`n^S>~4S06;=!1(xy6EKv!q+0SVE#eEz-MMR9*lez2 z+Kj_2tUTO3+)oE~j&A+oyrRjxR#`5m8CDB($7FogBeiOp%vcMD(H@N#N1Jf*)9Fq_ zmqGr}5;!Xx79NN~{GwO>wDUa#v#Hm=VTH)s9r z0uVGrO+7kEh6FX+@FOfAgg25A>p%R*ejt!p5QEk}uECK5VX6JpnU@~+_;lRIBqCspa~$|Lb`IsVgD0V`#*Ks|de%kQ-5 zO*OkcWxob97LvxS`_o~6X=U5&CiCuGLuI6*8<7v4(50PhXUU zRiElo$MT!huE3#<+~Cu#v7Y;WX#y|kHz~x3m7CQ+Am&U?Zne*B^H^aZlI&0>S^jm!(EvXkUm9_F0V4qIV z!V)Q`(T+RP`DKIe#UF2>rrn0cg}LOOXv%H>tGUHV@wPd_-cmDzPbkHU^aZNVE#oPD+0KNMI_I zCz|A}`|5}PyG-#R=JcUQK-hB^TER?(fdmR6K=|f^M*=Adgq)HR?U5JM{4DNJ@``VDhy1NPp7`9=db`_@v6(zNq&A&Eb<{b2 zu~nFT8us=Jgfd%BB=secvNQ&n+#f*juS4Be=ZNtLFpO+jZ0IT2;AyKGk${a1Q_a8kyIykkP@xvU~TeoR7Msnflf@0?GTLb;y#7DWg zJM>VaQjgIF9dy|+0yhto^UGz4&&uC5jXJlP+hd6+O_wpc@`cY`UmQ`M#QlYkK%n&l zbmDcH=}#RnO&$1fezZB6lv%fLFVmy1@4`VAl9#`7vz`SjCut_YEWBtwuZu|sGfSt` z{H7mV={u1}rpc>s;{F2`QsEs|5=Uf}O%EH~j+%)&d#D1C34#MvdWx!}^)?f%g1+Ym z1r@d5NezUClEuQL@W|`?>VeLsa&ql|m--hCENc4dba_x~VOexo%U9}et{I_|szCs~ zIu+;#Q+hiSxBkZ&)GrUJEczRs!|56?jV~_WdA_VZYp<%%tK^M|Q>3!;b16k9jH-I0 zd2|_WCc;2D3&AD%O_kv!r#jj#{OTeu@Lk_vHHiQmS_2eRO`FVFf7dMmF1h<$yEtR` zec`SwOOG9zNAw!N-3iV{6M(oSsKGDaibvK!8h2~@U+nqq`xt9!S18XWzH4B4ccUav zW9J@bLG6&nSI*zx^vQ4Z%^jOndo4V&^-fRRKu>>`fDX(qKcFw|I`3levvJ}WysFtU zu~gMba8y~j!mnKixoO#mT!r7*1LW!KU~B;@517V(FV8aFqbV~JMlAnx5zjE8BQ(Op zbF+vli^YYd$ke9PB}j9cKxu{1Ge^a7W74>)P!DZ$XgVLIiE@&oxVviDe|rWzm0EcZ zc}fsPef{wo;fmDT#59=3cN9(vf=B5E9gsqTrd@!um8b2~;(vPS_PRliu8&CBoi;N) zvJ6_i-2^VUrtwa5i0^$Cy#7vrw|6}Pa%r4py%|tGePx^*p2G5YFpbX!nlp8I zcn+v|Ta6gZ5ax9=!H=u&(QX~y_VA(;60C9JNvBA>0^4g$qq`JpV~kiT8hOoK{)%@pW# zcCjB@`0YZl@jq1h%x@zvw<{Od%MXwg{UK=AH09y2eEE`YXA62ecQ~!XM@Z6r?h0C= z>+qG&(FGV}{kfcKUzm65#Fv;}_GrJ#^!fEQ44K?^%y1U`w1s-u=G-<8;JJrvYBFw^ zeZUkYA?<$Q*) z3lcSZ!#i@-$;G?dGx(fi`+NW(M^4cJezxFqbt2ey2zA&R%}2rDO{M7F^$}&|(kH!3 z;yNiVXz-6c|EX}GvK2C_VOOV4eaYa z8#uO}__k8JCCV${8TnGx9(TQKdb>Rdg$&Lj!a?rm|5jb~%K!Y4fxz%*L6jcI>NCTy zUOFT^rw#2l9Eb6FKp#}n+7pm+d2^4lM7@C< zdn-|#?wmd|{Vmu~7uA#mYH%-0Va7_^?MWs4(Tm28dFLL=wtWg3t5|Kf}r_u!?5w<9G8{)Zef{c1sbH^B#-yK53LaA91LBN9I5D zbNEwPn&cU``}_ZRO5PxOdl-{db(p?QjmP#xJYf5yF4DvqNE)^ju(xuaj{zOW+%t8F&uD!} zAiM^AX-pS?V(;ic9kpU_ zkUKNA^CaqG%cxq8$783ur+Ll+KA3P?)plM;Bz4e%N$8ot5?6p~>X*Z*pI?pSPTZd}+;k zqE9jt=!17UgQ)PyWvqhU79TMu4^zuao>lrK9Kezf(XT}djRYQQw#=I4hsb;r+?y`tNWUNHHR1CaEsG-=l@qop%QM%~;(OwEHRSMBTW`O;YU&t3k0IQAOf> ztk$oAz7Wqzi!I|;HS>k2E<9(y6*{SBQvSHo{^yuhc<%3^BVK~9rP{>hHO|gx5E$`5JqC=JckY!B>=G!a0nXx)pouaTcWqtSuxDjdzFhxZc zc^7~X!W_|(&tR9>LoF*T*Lb~57~r&cy&>p_^DU0}gaZ?3(qCi>)f+qfCDwlaozpjk zS1v}H^RY-S5o;e!Mrskc2;cH>QRLRzTKn=32-8!Uf`}ct)lMz^%s+zd7f_GpJd*Z< zP|jm7r{0*5fDT0HqxoU~Q{KwRjp6G3wyWlbmUWJ{7ZeEvX6y3EQJ6uvG1H{8!+JzK zBwkL4s1-J4!&?AGT9Ikpm8ygu#3#4p2{t?l=XQTv!PZT1dv(1S za1cV}`5`PpAb~M@p)k{ZR)jjCU>Wg<)72QYgy-{=5|9q?-3=uY7Sf8=Q+>&T_xctq zEo<@0*B2_jDJi|_p$hb1I|Mysm^jWmmPb>Xi-23A1{!A5%{5^3De6n5%Q;vafjKHdH>!}Io=tHrwwtDM&a=r}sq!b|Ln1}jSZ?m%Jh zMZdENA=){^jeg5_U*bl6$b}nKdPcqWn>L@Tn>TXyf~Wuwr37TAg|ZbIumAM>6+)=L z%7;*-E+GM(Onsr-rjjq874_1EP@szM=(WJpIY+#2hoZ0V73TJucYcy0`mCp}Wlz;5Q}w-s%dHg`_F_3FMWB>pYPfV-QU zuKtv7YY~>8aI|3v06p+S_ysQ1zpgsMnNS!7I)2_b&)-oKe@g$PBzd!?eiITS`Rk%G z?-J6J=qDa^+e3kPHYnDS!tvuNdLxF*E~@%YYj*0-!{60R?dk=cL0EX~3pcKEr_QxQ$AH~EIJOTkEFfTxY1gq)o2%cv+k?39S%A-v) zpR@Gcoaop`ilC4en8LvqiB-R07)DE^Y2eYUxjttTc%7zN(k9QLYP!Iny0^7FI6>X- zw-x~}DWfYE2OE**1Y&4Vx&_p^WLqcu$xN;AlIre^>$`L~i0~bw^~O(M6IxSW#AbY0 z-D}VHe8$GG(-c(>O!1yLiYH$=k4A7<9z7W3K{(gPIOk1Hn-`NG7(VKu_t%XMQ98lh zj_vUtxZxb^#;3?jt~;u>DBc^_}s&IqpXP$l4aAc=>F0rRG>S z30QpP<09ECw+g^IA3#HZBMQjGy#j_41uJ!8ok-4Q`(J|RV3Wl7S>m|0mz_$GLPje0G9poeIL#T zF&*kIQ^57J!n@O?E_cd4?UqSqVlr|_F}x;U?!{oiwjE{@at0?qmtqUZ)8z=_u|(XE z7Xb>}18BI1OW-bSCeiv2-~iJsu2j6i{SFx~vR_<=QsEM)!p6rGdq(DfO7`fx9Mmi? z6`!^8n|+ph(VQHB)23-qOjazDg0Z z^b8pps3oeJJ6>*t3a-1d;e11G(d9+V7rxY6DsRqTA4p>=8;2xnf6R1wF=}sepw}Rq zwEi7=H?oquMJ4VHJ1e}N?va@E**=GAcwD=;-QDb8?WWNG3}3eVCKPC#dZOHoj}!+O zA+#af1cMScPZL~KR&F%y_&Q(@8Uaw_a6d6&PCq6FDn4&>5Gc?njBR0ns$wyZCmh_qRWDI*{m=(%r@SGwC%her6?9T~ zl*k{{NiIJ#5b_n6@>xMI&^T`$gg@-)MrrZN`iX5_i^33&70hAIz9EzDhm-zRt-_;( zj2De+EO=-I79}p$Z{X0GG@Wvz^-*kw_a7iw(nIH%2hvfY5rI!na%e)FX1<(49eq2~ z$+?u4=7n7XVZk6ThPkFU^&$$xn~M zKz{L(GED2^TIwgaL8L5^-T1ZILJ>NpRB3OY)3fHLaOJPi59R5xl|)`yzRx9;>R^eI z7`1z&t?5@XAzo>JysHKkmfs3U8N1jITgBX!H5W1i5)ysVV6>h=;S))VeFK9hulsva z{7`Oe$+y10J>ROv>ziqf3fI*$O%zTF>+Pa=Fbn7GW0wuK9C!71W)g!4*S<#cw&XlH zT0^{Gw;;k|0UO+|hYSy1MX)~@RmhTgOp*Cmk;j+jSP$iEU7J~ITU7bikw4KGH#LAQ zi(Ty$Yc^t5#fKmj03_JxBf7mHD!K%{DKpd%JXQeu1BKB~EfZlIv~8tzDY z0G4D#RdqJbS0{Ee5aO9^dzt*SP{rhQ*R29Wu>>?p3$}XnD}b4~xBrKjyepU#!qG*B z&q-P|a>j#M@0bpn&qw=^A8)%h z+h3c$ufQueT)uU$aH+hG8MQ_>$HVwB{q?s8V#JzKP2PfT6LcN|?C|dk88XcT6GPz?}V$l-;P&f!O!*{9E z&}X5(j(wx1Kuzxec0n##AqvxC;iW~i2%Xk-fI#d&Bwi#0O%8pr#2ua#i}6uPoR*2L zmO>?!=lhkuexk@)ocFRsmsgVG;}7y(;+39|ltUV_>mrR@-zKb;rBf{qLKYoX`T=`D z99=KrN)JeoU0j!Gme44u4o_0^T!BBXpq!kQzdp1Je`OR(i+0UGdZI?tsVCom`(qRw73ngIkRsp37hGposap+ zSqh4vqwD2y{HkZ_8zawUv67&0NnjL38ef*@o2%qD%t{E>yx3hJa1QCsp?H{Yq6(si zc7Lp*VZX3n<6v(wnCvfeW+OJ)PP#3)`bYjrBtlmt7k=Tj|3PuOaq8ZPka=7fU-6Eb zgCj~b8FJF+Ls&l96r7t`EZ0<6VMI)px9Xj7{3Q>#X-8tc9+Ob7<;BCC%?g6(a&prj ztmtEFfi-Z`2ej62&%QfkY2Inwu|32P(5({{G|)H*FV9jkGqh0!<*`RpIoAS(Pd#c= zA5-*wab{yR!3>D_D`bto3~Y)_B>Qw0-w33x5!fmhDOOAe|11GdBMdAgcMQ9B$+KlY zW^l`2$-XxJ4ed^ptdsjIr1eFhaWe0}Ikpxi8r-P3Jl)+sf0OXrAcT&_Wa4@{SawoB zw9yS701AApvC4?3*U_CM{D<}{CaS`5;S-G)0)FA>m5a-xFWnx~>5t*qB30#`+P$@3 zGV{l*F2rPcWRjq+G1^{3F_&;HX4)ir(*9OINBktb^^V_X>UeH?v&eG4Tyax++I`02 zX|}mg@d7`;eJp2+C-JH>YvQVK<=kN2k`@-!Ko4M`88GhtmxxhRKzpD}H18F^U*zdC z3KZC5jf17umuVm@Upr@A&t=KdS)zPzDX#OEqx7Kdpy0QiWLHO;vko>81`3kDGtwWn zZTYRFJvv!LWiTpF?eNFZhamG=L^`UQ;fa5`k@BGF@P&3m&x|4)>l$|L_xNkD_rUW% zN-kmit364sQ3ch?24C|&^wo%yQVDMJ!5%?s*lDwKYGCFkSK;%8iZ_+s@Xr>uJ2#wc z8L15#jZ#aG4QGW)1}eXPBIE6@hGuw{CD?@a8s`1H@Sq&E^tT>Nlkbe2^%m`e)kHP! z@Hw-wndEX>kFu{w()wkZW%EtkjW$>i#e6~~Ht5#z))*z{=dcz-lp+#+W>WBEgN)Ce z9UgQ&)ACS4If?y{OK)YxqCcrAUpe{G9vaUVpm|V73brlOVOQQYe0_(gaF1-%>tRj* zI4Ba#s4AP2(Lc?kdeZ*kR1{u1eU&}I+t1)K(Hlo6?v?uv@C+lOlo80^`!L)8=F&@5 zbEeMSYpAp}Z%|PV&UoaWY+QcS)i%iWgKb=7IC_F*hac?z@WQtObgvI3t4sbG*G4IF zyWOF}w&O(_w>+XdRh=TqbTnmpTQX411($h|C$_5}TQ(<*Mw7xA%QDki1nQLxqJ~O& zH|MKoAL>I9{NPEnLM*uHy%%1M{IEgF?A%l;Oo+CzRW`jC2JLpW%qD{%D&HcZG!5GY zEZN&BF$VAot-KH4Fn29QUDjN4%$^kEztR00K-QcX18x?t*hY)J>W0Z_PuTIEt;8m; zeVypvHmYe>gJ;QzOjOq(@_&BF+p>OKZAwW6FOa>de0x*;bGCcoj=VKnjUr~WpjX0S zp!j^(`&h!|KR#lzhO1uvi!}@b;Vi(^gfXOBqzg=P>_j zx+J%x{I38F1YyYkZ061;=2w=pV}jY5z$0Lx)^arqtj;7RQF%1l_veomuGKsD*TaFIgrNb_`HrvzdJ~Ztw58gUTF4p-9Mr`AjlN}VH~SA>v12q zvvFaz9MMt}{%$+z$z;u>@^xY);XWnC+1b8XwahFxY&xY4lArj`jq!BvX9eTWWxm8J zEcV4>({8Bx3_jjIjVykvsJ|{A4%4P3p4o?WsX{nvK5DuBM=`g+O2Jh66*ZHnYePd5 zSG(xnlU8}4Z7>k^*%S-^j&IUyef-aQ|BY>dDTB>2$_Q$j%|~r@DCH!p_sKVcY|4kMO8@reL0c7F zV2>hN+z2JFtL9LQgBDHjsTInAnp+ux4JGKFOz0S#EcA2)Vb0%EXM1f|4jx-w(efOe&|B z6wZ+Csmg58-$l8|@`i=q4(&JY*{cv;ZR!^0(^y%8!_5yZd0tr54lUSns-C#l=r|Nh1d`WI9vi7v%_yb%>ZlNu2>5vv^dV8h+^T@@iLm{}=l^W@ z00cb-979m_%#h@rZ~NB0nh^HfyC_}Ur`{mo=42j4sEFYj)Z3*p8LkP@w2&GLz`!WZ zI(e!q(`{0359u;|V)4aL6s6kEg9|e^0<`Mm8V{QFpJMzwBa}~zpi-RRIg}_wDg7Yb zp}AP}UTcY^krmSeahpPX*5Y3Jvasn^9QRL@>WVH9>9BN$5qem!sc_{x)cjyP9YZb8 z91lab|FQ_g*dqGHA49AXsPq`&jxx|p==V$ZR&XClYeO_+oeRB=$8w{vYQO2`Hx@+E zvHJA(Q5^M7x<+34yQgVQ3sm}lwQltPWHx9ZlYMMe1PN0!iZb`Q<6CI`z8U=Jg%i$m z&o4eJ5lqM@y6=`FW)*H9?iFfEg!v4I1(r2go7Sm)k)9@UQ5-`UMKm1_G zDkCa#LItdsaYNLL=As5g-zED0u?VFb8L=(%_Vz}iq|==cLK%#Dl=RZbl zZ8yzzhI#}ncX_uuF-G=3w^EM)!6fGYb175$iy-hTZ}OD(Us?P=n|(%6n{<+}87S%g hmQnY&1QF03NIG#>MSRPm>Ow=kl;qXqDrDXV|33<8cgp|( literal 0 HcmV?d00001 From d051d776a779192a95260f02c2a6a6c8d6cc0020 Mon Sep 17 00:00:00 2001 From: Justina Petraityte Date: Tue, 16 Jul 2024 11:29:31 +0100 Subject: [PATCH 2/3] updates based on the feedback --- docs/composedb/set-up-your-environment.mdx | 18 ++++++++++++++++-- .../guides/ceramic-nodes/running-cloud.md | 8 +++----- .../protocol/js-ceramic/networking/networks.md | 1 + .../js-ceramic/nodes/running-a-node.md | 10 +++++----- docs/protocol/js-ceramic/overview.md | 2 +- 5 files changed, 26 insertions(+), 13 deletions(-) diff --git a/docs/composedb/set-up-your-environment.mdx b/docs/composedb/set-up-your-environment.mdx index 4a7ef9eb..db104576 100644 --- a/docs/composedb/set-up-your-environment.mdx +++ b/docs/composedb/set-up-your-environment.mdx @@ -24,7 +24,12 @@ There are a few ways to set up your environment. Choose the one that best fits y **Install and start the `ceramic-one` binary** All of the configuration options listed above **require a `ceramic-one` binary up and running**, which provides a data network access. You can run `ceramic-one` on your -local machine using two simple steps: +local machine using two simple steps listed below. + +:::note +The instructions below cover the steps for the MacOS-based systems. If you are running on a Linux-based system, you can find the +instructions [here](https://github.com/ceramicnetwork/rust-ceramic?tab=readme-ov-file#linux---debian-based-distributions). +::: 1. Install the component using [Homebrew](https://brew.sh/): @@ -38,11 +43,20 @@ ceramic-one daemon ``` :::note -By default, the command above will spin off a node which will connect to a `clay-testnet`. You can change this behaviour by providing a `--network` flag and specifying a network of your choice. For example: +By default, the command above will spin off a node which connects to a `clay-testnet` network. You can change this behaviour by providing a `--network` flag and specifying a network of your choice. For example: ```ceramic-one daemon --network clay-testnet``` ::: + +By default `ceramic-one` will store its data in the current directory. You can configure this behaviour by +specifying the `--store-dir`and `--p2p-key-dir` arguments. For example: + +```bash +ceramic-one daemon --store-dir ~/.ceramic-one --p2p-key-dir ~/.ceramic-one +``` + + With `ceramic-one` binary up and running you can move on with the ComposeDB installation and configuration method of your choice. diff --git a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md index b0a1d28a..d3e7a97b 100644 --- a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md +++ b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md @@ -30,16 +30,14 @@ instructions can be applied to the vast majority of other cloud providers like A **Supported Operating Systems:** - Linux -- Mac -- Windows :::note -For Windows, Windows Subsystem for Linux 2 (WSL2) is strongly recommended. Using the Windows command line is not portable and can cause compatibility issue when running the same configuration on a different operating system (e.g. in a Linux-based cloud deployment). +At the moment, developers are provided with Linux-based docker images for cloud deployment. ::: **Compute requirements:** -You’ll need sufficient compute resources to power `ceramic-one`, 'js-ceramic' and `PostgreSQL`. Below are the recommended requirements: +You’ll need sufficient compute resources to power `ceramic-one`, `js-ceramic` and `PostgreSQL`. Below are the recommended requirements: - 4 vCPUs - 8GB RAM @@ -153,7 +151,7 @@ By default, your Ceramic node will connect to the Ceramic `clay-testnet`. In or All state in this configuration is ephemeral, for persistence use docker-compose. -1. Start rust-ceramic using the host network +1. Start ceramic-one using the host network ```json docker run --network=host \ diff --git a/docs/protocol/js-ceramic/networking/networks.md b/docs/protocol/js-ceramic/networking/networks.md index 5acb8159..95dc0359 100644 --- a/docs/protocol/js-ceramic/networking/networks.md +++ b/docs/protocol/js-ceramic/networking/networks.md @@ -19,6 +19,7 @@ An overview of the various Ceramic networks available today: | Mainnet | mainnet | /ceramic/mainnet | Ethereum Mainnet (EIP155:1) | Public | | Clay Testnet | testnet-clay | /ceramic/testnet-clay | Ethereum Gnosis Chain | Public | | Dev Unstable | dev-unstable | /ceramic/dev-unstable | Ethereum Goerli Testnet | Public | +| Local | local | /ceramic/local-$(randomNumber) | Ethereum by Truffle Ganache | Private | | In-memory | inmemory | | None | Private | :::note diff --git a/docs/protocol/js-ceramic/nodes/running-a-node.md b/docs/protocol/js-ceramic/nodes/running-a-node.md index 45a76d46..b882c836 100644 --- a/docs/protocol/js-ceramic/nodes/running-a-node.md +++ b/docs/protocol/js-ceramic/nodes/running-a-node.md @@ -71,20 +71,20 @@ Check the `js-ceramic` service’s availability with the healthcheck endpoint curl http://localhost:7007/api/v0/node/healthcheck ``` -Check the `rust-ceramic` service’s availability with the liveness endpoint +Check the `ceramic-one` service’s availability with the liveness endpoint ```json -curl http://127.0.0.1:5001/ceramic/liveness +curl http://127.0.0.1:5101/ceramic/liveness ``` #### Metrics -Both `rust-ceramic` and `js-ceramic` have prometheus compatible endpoints available. +Both `ceramic-one` and `js-ceramic` have prometheus compatible endpoints available. -`rust-ceramic` is enabled by default +`ceramic-one` is enabled by default ```jsx -curl http://127.0.0.1:9464/metrics # rust-ceramic metrics +curl http://127.0.0.1:9464/metrics # ceramic-one metrics ``` js-ceramic monitoring configuration is described [here](https://developers.ceramic.network/docs/composedb/guides/composedb-server/server-configurations#prometheus-endpoint0). \ No newline at end of file diff --git a/docs/protocol/js-ceramic/overview.md b/docs/protocol/js-ceramic/overview.md index fca3b8c2..23fec27b 100644 --- a/docs/protocol/js-ceramic/overview.md +++ b/docs/protocol/js-ceramic/overview.md @@ -5,7 +5,7 @@ Ceramic is a decentralized event streaming protocol that enables developers to b The latest release of Ceramic has introduced a new Rust-based implementation of Ceramic protocol which offers performance and stability improvements as well as a new data synchronisation protocol called Recon. Developers, building on Ceramic network will be using two main components: - `js-ceramic` component which provides the API interface for Ceramic applications -- `rust-ceramic` component which provides Ceramic data network access (contains the implementation of Recon protocol). +- `ceramic-one` component which provides Ceramic data network access (contains the implementation of Recon protocol).
![protocol-overview](/img/protocol.png) From 646f507ff74f8e31ce42bb1bf8201f63a7364f84 Mon Sep 17 00:00:00 2001 From: Justina Petraityte Date: Tue, 16 Jul 2024 13:12:40 +0100 Subject: [PATCH 3/3] naming change for simpledeploy --- .../js-ceramic/guides/ceramic-nodes/running-cloud.md | 2 +- .../js-ceramic/guides/ceramic-nodes/running-locally.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md index d3e7a97b..ad8e75dc 100644 --- a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md +++ b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-cloud.md @@ -116,7 +116,7 @@ You will know that your deployment is up and running when all of the processes h ```bash NAME READY STATUS RESTARTS AGE js-ceramic-0 0/1 Running 0 77s -rust-ceramic-0 1/1 Running 0 77s +ceramic-one-0 1/1 Running 0 77s postgres-0 1/1 Running 0 77s ``` diff --git a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md index acc5e812..a6b4b8fc 100644 --- a/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md +++ b/docs/protocol/js-ceramic/guides/ceramic-nodes/running-locally.md @@ -77,7 +77,7 @@ ceramic config set ceramicHost 'https://yourceramicnode.com' ``` ## Monitoring -You can always check if `js-ceramic` and `rust-ceramic` components are available by running the commands listed below. +You can always check if `js-ceramic` and `ceramic-one` components are available by running the commands listed below. ### `js-ceramic` service's availability @@ -87,10 +87,10 @@ Check the `js-ceramic` service’s availability with the healthcheck endpoint: curl http://localhost:7007/api/v0/node/healthcheck ``` -### `rust-ceramic` service's availability +### `ceramic-one` service's availability -Check the rust-ceramic service’s availability with the liveness endpoint: +Check the ceramic-one service’s availability with the liveness endpoint: ```json -curl http://127.0.0.1:5001/ceramic/liveness +curl http://127.0.0.1:5101/ceramic/liveness ``` \ No newline at end of file