docs: Set up docs site infra and refactor existing docs into the site. (#127)

Co-authored-by: Kirk Rodrigues <2454684+kirkrodrigues@users.noreply.github.com>

Author: junhaoliao

.gitignore

@@ -2,6 +2,8 @@
 /node_modules
 
 # Build
+/.task
+/build
 /dist
 
 # For bundle size analysis

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "tools/yscope-dev-utils"]
+	path = tools/yscope-dev-utils
+	url = https://github.com/y-scope/yscope-dev-utils.git

README.md

@@ -4,83 +4,52 @@
 ![Open feature requests][feature-requests-shield]
 ![CLP on Zulip][zulip-shield]
 
-`yscope-log-viewer` is a tool that can be used to view log files. It currently
-supports [CLP][clp-repo]'s compressed log files (IR streams) and JSON log files.
-The viewer can be used to navigate the log file, filter by log level, view the
-logs with syntax highlighting, and generate direct links to specific log events.
+The YScope Log Viewer is a tool that can be used to view log files. It currently supports
+[CLP][clp-repo]'s compressed log files (IR streams) and JSON log files. The viewer can be used to
+navigate the log file, filter by log level, view the logs with syntax highlighting, and generate
+direct links to specific log events.
 
-See the [online demo](#online-demo) section to try out the log viewer on a
-sample log file. To set up a local server, follow the
-[build guide](docs/dev-guide/building.md).
+See the [demo](#demo) section to try the log viewer on a sample log file. Or you can generate your
+own compressed log files using one of our [libraries][docs-site/generating-ir-stream-logs].
 
-See the [features in development](#features-in-development) section for upcoming
-features.
+Want to report a bug or request a feature? Check out the [feedback](#providing-feedback) section.
+A list of [features in development][docs-site/features-in-development] is available on our
+[docs site][docs-site].
 
-# Online Demo
+# Demo
+
+A demo of the viewer available [here][online-demo].
 
-* A demo of the log viewer can be found [here][online-demo].
 * The demo loads a Hadoop YARN log file from the [hive-24hrs] log dataset.
   * More info on the dataset and other datasets can be found [here][datasets].
-* To open an IR stream, drag and drop it onto the log viewer or use the open
-  file dialog.
-
-# Generating IR Stream Logs
-
-IR stream log files can currently be generated using these libraries:
-
-* [Log4j Logging Library][log4j1-appenders]
-* [Logback Logging Library][logback-appenders]
-* [Python Logging Library][clp-loglib-py]
-* Golang Logging Library (in development)
-
-# How does it work?
+* To open an IR stream, drag and drop it onto the log viewer or use the open file dialog.
 
-The log viewer is written using the ReactJS framework and uses the open source
-[monaco-editor] and [clp-ffi-js].
+# Docs
 
-The viewer spawns a worker to assist with computationally intensive tasks such
-as:
+The log viewer's docs are available [here][docs-site].
 
-* Deserializing the file and creating an index of logs events.
-* Paginating the indexed logs.
-* Decoding the deserialized log events into plain text.
+# Providing feedback
 
-Tasks are passed to the worker as needed and changes are rendered in the UI.
-
-# Providing Feedback
-
-You can use GitHub issues to [report a bug][report-bug] or
-[request a feature][request-feature].
+You can use GitHub issues to [report a bug][bug-report] or [request a feature][feature-req].
 
 Join us on [Zulip][zulip] to chat with developers and other community members.
 
 # Contributing
 
-See the docs in our [developer guide](docs/dev-guide).
-
-# Features in Development
-
-* Pretty printing to enhance the readability of structured data in the logs.
-* A dashboard to visualize the distribution of log types and log levels.
-* Support for plain text, archived log files, and other requested formats.
-* Searching within a file, multiple files, or within provided time ranges.
-* Infinite scrolling instead of pagination.
-* Log correlation with sync by timestamp across multiple editors.
-* Automatic conversion of text log files to IR stream format in the browser.
-* Deployment of components via NPM.
+See the docs in our [developer guide][docs-site/dev-guide].
 
+[bug-report]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=bug&template=bug-report.yml
 [bugs-shield]: https://img.shields.io/github/issues/y-scope/yscope-log-viewer/bug?label=bugs
-[clp-ffi-js]: https://github.com/y-scope/clp-ffi-js
-[clp-loglib-py]: https://github.com/y-scope/clp-loglib-py
 [clp-repo]: https://github.com/y-scope/clp
 [datasets]: https://docs.yscope.com/clp/main/user-guide/resources-datasets
+[docs-site]: https://docs.yscope.com/yscope-log-viewer/main/
+[docs-site/building]: https://docs.yscope.com/yscope-log-viewer/main/dev-guide/building-getting-started
+[docs-site/dev-guide]: https://docs.yscope.com/yscope-log-viewer/main/dev-guide/index
+[docs-site/features-in-development]: https://docs.yscope.com/yscope-log-viewer/main/index#features-in-development
+[docs-site/generating-ir-stream-logs]: https://docs.yscope.com/yscope-log-viewer/main/index#generating-ir-stream-logs
+[feature-req]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=enhancement&template=feature-request.yml
 [feature-requests-shield]: https://img.shields.io/github/issues/y-scope/yscope-log-viewer/enhancement?label=feature-requests
-[hive-24hrs]: https://zenodo.org/record/7094921#.Y5JbH33MKHs
-[log4j1-appenders]: https://github.com/y-scope/log4j1-appenders
-[logback-appenders]: https://github.com/y-scope/logback-appenders
-[monaco-editor]: https://microsoft.github.io/monaco-editor/
+[hive-24hrs]: https://zenodo.org/records/7094921#.Y5JbH33MKHs
 [online-demo]: https://y-scope.github.io/yscope-log-viewer/?filePath=https://yscope.s3.us-east-2.amazonaws.com/sample-logs/yarn-ubuntu-resourcemanager-ip-172-31-17-135.log.1.clp.zst
-[report-bug]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=bug&template=bug-report.yml
-[request-feature]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=enhancement&template=feature-request.yml
 [zulip]: https://yscope-clp.zulipchat.com/
 [zulip-shield]: https://img.shields.io/badge/zulip-yscope--clp%20chat-1888FA?logo=zulip

docs/README.md

@@ -0,0 +1,46 @@
+# Docs
+
+This directory contains the files necessary to generate a Sphinx-based documentation website for
+this project:
+
+* `conf` - Configuration files
+* `src` - The actual docs
+
+## Requirements
+
+* [Git LFS][git-lfs]
+  * We use Git LFS to store the images in the docs; this is to avoid significant increases in
+    the size of repo as we add and update images.
+* [Node.js] >= 16 to be able to [view the output](#viewing-the-output)
+* Python 3.10 or later
+* [Task] >= 3.38.0
+
+## Build commands
+
+* Build the site incrementally:
+
+  ```shell
+  task docs:site
+  ```
+
+  * The output of the build will be in `../build/docs/html`.
+
+* Clean up the build:
+
+  ```shell
+  task docs:clean
+  ```
+
+## Viewing the output
+
+```shell
+task docs:serve
+```
+
+The command above will install [http-server] and serve the built docs site; `http-server` will print
+the address it binds to (usually http://localhost:8080).
+
+[git-lfs]: https://git-lfs.com
+[http-server]: https://www.npmjs.com/package/http-server
+[Node.js]: https://nodejs.org/en/download/current
+[Task]: https://taskfile.dev/

docs/conf/conf.py

@@ -0,0 +1,78 @@
+# -- Project information -------------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "YScope Log Viewer"
+
+# NOTE: We don't include a period after "Inc" since the theme adds one already.
+copyright = "2022-2024 YScope Inc"
+
+# -- General configuration -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "myst_parser",
+    "sphinx_copybutton",
+    "sphinx_design",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.viewcode",
+    "sphinxcontrib.mermaid",
+]
+
+# -- MyST extensions -----------------------------------------------------------
+# https://myst-parser.readthedocs.io/en/stable/syntax/optional.html
+myst_enable_extensions = [
+    "attrs_block",
+    "colon_fence",
+]
+
+myst_heading_anchors = 4
+
+# -- Sphinx autodoc options ----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#configuration
+
+autoclass_content = "class"
+autodoc_class_signature = "separated"
+autodoc_typehints = "description"
+
+# -- HTML output options -------------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_favicon = "https://docs.yscope.com/_static/favicon.ico"
+html_title = project
+html_show_copyright = True
+
+html_static_path = ["../src/_static"]
+
+html_theme = "pydata_sphinx_theme"
+
+# -- Theme options -------------------------------------------------------------
+# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/layout.html
+
+html_theme_options = {
+    "footer_start": ["copyright"],
+    "footer_center": [],
+    "footer_end": ["theme-version"],
+    "navbar_start": ["navbar-logo"],
+    "navbar_end": ["navbar-icon-links", "theme-switcher"],
+    "primary_sidebar_end": [],
+    "secondary_sidebar_items": ["page-toc", "edit-this-page"],
+    "show_prev_next": False,
+    "use_edit_page_button": True,
+}
+
+# -- Theme source buttons ------------------------------------------------------
+# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/source-buttons.html
+
+html_context = {
+    "github_user": "y-scope",
+    "github_repo": "yscope-log-viewer",
+    "github_version": "main",
+    "doc_path": "docs/src",
+}
+
+# -- Theme custom CSS and JS ---------------------------------------------------
+# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/static_assets.html
+
+
+def setup(app):
+    app.add_css_file("custom.css")

docs/dev-guide/building.md

@@ -0,0 +1,6 @@
+myst-parser>=4.0.0
+pydata-sphinx-theme>=0.16.0
+sphinx_design>=0.6.1
+sphinx-copybutton>=0.5.2
+sphinx>=8.1.3
+sphinxcontrib-mermaid>=1.0.0

docs/dev-guide/state-and-stateRef.md

@@ -0,0 +1,32 @@
+html[data-theme="dark"], html[data-theme="light"] {
+    --pst-color-primary: #3399ff;
+    --pst-color-secondary: #9580ff;
+}
+
+a {
+    text-decoration: none;
+}
+
+a:hover {
+    text-decoration: underline;
+}
+
+/*
+Use the bottom border that's used for indicating the current page as the hover style (for a more
+cohesive look).
+NOTE: This selector matches the one in pydata-sphinx-theme
+pydata/pydata-sphinx-theme@v0.14.4/src/pydata_sphinx_theme/assets/styles/sections/_header.scss#L86
+*/
+.bd-header .navbar-nav li a.nav-link:hover {
+    border-bottom: max(3px,.1875rem,.12em) solid var(--pst-color-secondary);
+    text-decoration: none;
+}
+
+/*
+Remove margin from sidebar-primary-items__end so that we don't have an unnecessary scrollbar. We're
+not using the end items currently.
+*/
+.bd-sidebar-primary .sidebar-primary-items__end {
+    margin-top: 0;
+    margin-bottom: 0;
+}