Merge branch 'main' into gs/list_unit_tests

.changes/unreleased/Features-20240122-145854.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Enable unit testing versioned models
+time: 2024-01-22T14:58:54.251484-05:00
+custom:
+  Author: gshank
+  Issue: "9344"

.changes/unreleased/Fixes-20240124-142522.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Fix retry command run from CLI
+time: 2024-01-24T14:25:22.846199-08:00
+custom:
+  Author: ChenyuLInx
+  Issue: "9444"

.changes/unreleased/Under the Hood-20240109-091856.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: dbt Labs OSS standardization of docs and templates.
+time: 2024-01-09T09:18:56.686698+11:00
+custom:
+  Author: tonayya
+  Issue: "9252"

.changes/unreleased/Under the Hood-20240123-142256.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Move WritableManifest + Documentation to dbt/artifacts
+time: 2024-01-23T14:22:56.488252-05:00
+custom:
+  Author: michelleark
+  Issue: 9378 9379

.changes/unreleased/Under the Hood-20240125-095453.yaml

@@ -0,0 +1,6 @@
+kind: Under the Hood
+body: Define Macro and Group resources in dbt/artifacts
+time: 2024-01-25T09:54:53.974332-05:00
+custom:
+  Author: michelleark
+  Issue: 9381 9382

.github/ISSUE_TEMPLATE/implementation-ticket.yml

@@ -62,6 +62,6 @@ body:
     attributes:
       label: Context
       description: |
-        Provide the "why", motivation, and alternative approaches considered -- linking to previous refinement issues, spikes, Notion docs as appropriate
+        Provide the "why", motivation, and alternative approaches considered -- linking to previous refinement issues, spikes and documentation as appropriate
     validations:
       required: false

.github/workflows/release.yml

@@ -70,7 +70,7 @@ jobs:
     name: Log Inputs
     runs-on: ubuntu-latest
     outputs:
-      starting_sha: ${{ steps.set_sha.outputs.changelog_path }}
+      starting_sha: ${{ steps.set_sha.outputs.starting_sha }}
     steps:
       - name: "[DEBUG] Print Variables"
         run: |

CONTRIBUTING.md

@@ -10,6 +10,7 @@
 6. [Debugging](#debugging)
 7. [Adding or modifying a changelog entry](#adding-or-modifying-a-changelog-entry)
 8. [Submitting a Pull Request](#submitting-a-pull-request)
+9. [Troubleshooting Tips](#troubleshooting-tips)
 
 ## About this document
 
@@ -44,9 +45,7 @@ If you are not a member of the `dbt-labs` GitHub organization, you can contribut
 
 ### dbt Labs contributors
 
-If you are a member of the `dbt-labs` GitHub organization, you will have push access to the `dbt-core` repo. Rather than forking `dbt-core` to make your changes, just clone the repository, check out a new branch, and push directly to that branch. Branch names should be fixed by `CT-XXX/` where:
-* CT stands for 'core team'
-* XXX stands for a JIRA ticket number
+If you are a member of the `dbt-labs` GitHub organization, you will have push access to the `dbt-core` repo. Rather than forking `dbt-core` to make your changes, just clone the repository, check out a new branch, and push directly to that branch.
 
 ## Setting up an environment
 
@@ -221,10 +220,12 @@ You don't need to worry about which `dbt-core` version your change will go into.
 
 ## Submitting a Pull Request
 
-Code can be merged into the current development branch `main` by opening a pull request. A `dbt-core` maintainer will review your PR. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code.
+Code can be merged into the current development branch `main` by opening a pull request. If the proposal looks like it's on the right track, then a `dbt-core` maintainer will triage the PR and label it as `ready_for_review`. From this point, two code reviewers will be assigned with the aim of responding to any updates to the PR within about one week. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code. Once merged, your contribution will be available for the next release of `dbt-core`.
 
 Automated tests run via GitHub Actions. If you're a first-time contributor, all tests (including code checks and unit tests) will require a maintainer to approve. Changes in the `dbt-core` repository trigger integration tests against Postgres. dbt Labs also provides CI environments in which to test changes to other adapters, triggered by PRs in those adapters' repositories, as well as periodic maintenance checks of each adapter in concert with the latest `dbt-core` code changes.
 
 Once all tests are passing and your PR has been approved, a `dbt-core` maintainer will merge your changes into the active development branch. And that's it! Happy developing :tada:
 
+## Troubleshooting Tips
+
 Sometimes, the content license agreement auto-check bot doesn't find a user's entry in its roster. If you need to force a rerun, add `@cla-bot check` in a comment on the pull request.

README.md

@@ -31,7 +31,7 @@ These select statements, or "models", form a dbt project. Models frequently buil
 
 ## Reporting bugs and contributing code
 
-- Want to report a bug or request a feature? Let us know on [Slack](http://community.getdbt.com/), or open [an issue](https://github.com/dbt-labs/dbt-core/issues/new)
+- Want to report a bug or request a feature? Let us know and open [an issue](https://github.com/dbt-labs/dbt-core/issues/new/choose)
 - Want to help us build dbt? Check out the [Contributing Guide](https://github.com/dbt-labs/dbt-core/blob/HEAD/CONTRIBUTING.md)
 
 ## Code of Conduct

core/dbt/artifacts/resources/__init__.py

@@ -0,0 +1,8 @@
+from dbt.artifacts.resources.base import BaseResource
+
+# alias to latest resource definitions
+from dbt.artifacts.resources.v1.documentation import Documentation
+from dbt.artifacts.resources.v1.macro import Macro, MacroDependsOn, MacroArgument
+from dbt.artifacts.resources.v1.docs import Docs
+from dbt.artifacts.resources.v1.group import Group
+from dbt.artifacts.resources.v1.owner import Owner

core/dbt/artifacts/resources/base.py

@@ -0,0 +1,15 @@
+from dataclasses import dataclass
+from dbt_common.dataclass_schema import dbtClassMixin
+from dbt_common.contracts.util import Replaceable
+
+from dbt.artifacts.resources.types import NodeType
+
+
+@dataclass
+class BaseResource(dbtClassMixin, Replaceable):
+    name: str
+    resource_type: NodeType
+    package_name: str
+    path: str
+    original_file_path: str
+    unique_id: str

core/dbt/artifacts/resources/types.py

@@ -0,0 +1,54 @@
+from dbt_common.dataclass_schema import StrEnum
+
+
+class AccessType(StrEnum):
+    Private = "private"
+    Protected = "protected"
+    Public = "public"
+
+    @classmethod
+    def is_valid(cls, item):
+        try:
+            cls(item)
+        except ValueError:
+            return False
+        return True
+
+
+class NodeType(StrEnum):
+    Model = "model"
+    Analysis = "analysis"
+    Test = "test"
+    Snapshot = "snapshot"
+    Operation = "operation"
+    Seed = "seed"
+    # TODO: rm?
+    RPCCall = "rpc"
+    SqlOperation = "sql_operation"
+    Documentation = "doc"
+    Source = "source"
+    Macro = "macro"
+    Exposure = "exposure"
+    Metric = "metric"
+    Group = "group"
+    SavedQuery = "saved_query"
+    SemanticModel = "semantic_model"
+    Unit = "unit_test"
+    Fixture = "fixture"
+
+    def pluralize(self) -> str:
+        if self is self.Analysis:
+            return "analyses"
+        elif self is self.SavedQuery:
+            return "saved_queries"
+        return f"{self}s"
+
+
+class RunHookType(StrEnum):
+    Start = "on-run-start"
+    End = "on-run-end"
+
+
+class ModelLanguage(StrEnum):
+    python = "python"
+    sql = "sql"

core/dbt/artifacts/resources/v1/docs.py

@@ -0,0 +1,10 @@
+from dataclasses import dataclass
+from dbt_common.dataclass_schema import dbtClassMixin
+from dbt_common.contracts.util import Replaceable
+from typing import Optional
+
+
+@dataclass
+class Docs(dbtClassMixin, Replaceable):
+    show: bool = True
+    node_color: Optional[str] = None

core/dbt/artifacts/resources/v1/documentation.py

@@ -0,0 +1,11 @@
+from dataclasses import dataclass
+from typing import Literal
+
+from dbt.artifacts.resources.base import BaseResource
+from dbt.artifacts.resources.types import NodeType
+
+
+@dataclass
+class Documentation(BaseResource):
+    resource_type: Literal[NodeType.Documentation]
+    block_contents: str

core/dbt/artifacts/resources/v1/group.py

@@ -0,0 +1,13 @@
+from dataclasses import dataclass
+from typing import Literal
+
+from dbt.artifacts.resources.base import BaseResource
+from dbt.artifacts.resources.types import NodeType
+from dbt.artifacts.resources.v1.owner import Owner
+
+
+@dataclass
+class Group(BaseResource):
+    name: str
+    owner: Owner
+    resource_type: Literal[NodeType.Group]

core/dbt/artifacts/resources/v1/macro.py

@@ -0,0 +1,40 @@
+from dataclasses import dataclass, field
+import time
+from typing import Literal, List, Dict, Optional, Any
+
+from dbt_common.contracts.util import Replaceable
+from dbt_common.dataclass_schema import dbtClassMixin
+from dbt.artifacts.resources.base import BaseResource
+from dbt.artifacts.resources.types import NodeType, ModelLanguage
+from dbt.artifacts.resources.v1.docs import Docs
+
+
+@dataclass
+class MacroArgument(dbtClassMixin):
+    name: str
+    type: Optional[str] = None
+    description: str = ""
+
+
+@dataclass
+class MacroDependsOn(dbtClassMixin, Replaceable):
+    macros: List[str] = field(default_factory=list)
+
+    # 'in' on lists is O(n) so this is O(n^2) for # of macros
+    def add_macro(self, value: str):
+        if value not in self.macros:
+            self.macros.append(value)
+
+
+@dataclass
+class Macro(BaseResource):
+    macro_sql: str
+    resource_type: Literal[NodeType.Macro]
+    depends_on: MacroDependsOn = field(default_factory=MacroDependsOn)
+    description: str = ""
+    meta: Dict[str, Any] = field(default_factory=dict)
+    docs: Docs = field(default_factory=Docs)
+    patch_path: Optional[str] = None
+    arguments: List[MacroArgument] = field(default_factory=list)
+    created_at: float = field(default_factory=lambda: time.time())
+    supported_languages: Optional[List[ModelLanguage]] = None

core/dbt/artifacts/resources/v1/owner.py

@@ -0,0 +1,11 @@
+from dataclasses import dataclass
+from typing import Optional
+
+from dbt_common.contracts.config.properties import AdditionalPropertiesAllowed
+from dbt_common.contracts.util import Replaceable
+
+
+@dataclass
+class Owner(AdditionalPropertiesAllowed, Replaceable):
+    email: Optional[str] = None
+    name: Optional[str] = None

core/dbt/artifacts/schemas/catalog/__init__.py

@@ -0,0 +1,2 @@
+# alias to latest
+from dbt.artifacts.schemas.catalog.v1.catalog import *  # noqa

core/dbt/artifacts/catalog.py → core/dbt/artifacts/schemas/catalog/v1/catalog.py

@@ -5,7 +5,7 @@
 from dbt_common.dataclass_schema import dbtClassMixin
 from dbt_common.utils.formatting import lowercase
 from dbt_common.contracts.util import Replaceable
-from dbt.artifacts.base import ArtifactMixin, BaseArtifactMetadata, schema_version
+from dbt.artifacts.schemas.base import ArtifactMixin, BaseArtifactMetadata, schema_version
 
 Primitive = Union[bool, str, float, None]
 PrimitiveDict = Dict[str, Primitive]

core/dbt/artifacts/schemas/freshness/__init__.py

@@ -0,0 +1 @@
+from dbt.artifacts.schemas.freshness.v3.freshness import *  # noqa

core/dbt/artifacts/freshness.py → core/dbt/artifacts/schemas/freshness/v3/freshness.py

@@ -2,8 +2,13 @@
 from typing import Dict, Any, Sequence, List, Union, Optional
 from datetime import datetime
 
-from dbt.artifacts.results import ExecutionResult, FreshnessStatus, NodeResult, TimingInfo
-from dbt.artifacts.base import ArtifactMixin, VersionedSchema, schema_version, BaseArtifactMetadata
+from dbt.artifacts.schemas.results import ExecutionResult, FreshnessStatus, NodeResult, TimingInfo
+from dbt.artifacts.schemas.base import (
+    ArtifactMixin,
+    VersionedSchema,
+    schema_version,
+    BaseArtifactMetadata,
+)
 from dbt_common.dataclass_schema import dbtClassMixin, StrEnum
 from dbt_common.exceptions import DbtInternalError
 

core/dbt/artifacts/schemas/manifest/__init__.py

@@ -0,0 +1,2 @@
+# alias to latest
+from dbt.artifacts.schemas.manifest.v12.manifest import *  # noqa