More docs

dev_notes/todo.txt

@@ -7,8 +7,12 @@
 			Should not be an "include" on file level, because that causes too much ambiguity about how this should be saved if overwritten.
 			Instead, make it an explicit operation accessible from API (thus MobiView2 and/or mobipy).
 			void patch(Data_Set *target, Data_Set *source); // Or something like that.
+				Not that straightforward due to
+					Can't just copy over data, need to update all internal cross-references.
+						Maybe unless you patch from a text file and parse it from scratch instead of patching with an already parsed object.
+					Conflicts must be handled. For instance, must check that index sets that are in both data sets agree.
 
-			This is useful because often you calibrate the catchment separately (runs much faster alone), then want to combine the new results with the full basin model.
+			Reason why we want this: often you calibrate the catchment separately (runs much faster alone), then want to combine the new results with the full basin model.
 				Is currently done with text copy pasting, but can be a bit tiresome.
 
 		We need an all_in_flux and all_out_flux or similar.
@@ -18,9 +22,7 @@
 		If you load different modules using the same load name, you don't get a name clash!
 			This is as intended if the two loads are identical (so that you can extend two models that load the same module)
 			Fixed if the module template is different, but should also be fixed when template is same but load arguments are different.
-
-		When a parameter changes name, the old one is not deleted from the par file.
-			Should be fixed, not tested.
+				Complexity because e.g. two 'loc' arguments could be different declarations but have the same "value". Same for any inlined declaration.
 
 		Document how to use connections, and some more of the advanced concepts.
 
@@ -29,18 +31,9 @@
 
 		mobipy
 
-			Can't load mobipy from Anaconda on Windows
+			Can't load mobipy from Anaconda on Windows (dll doesn't load properly)
 				It is probably more complex than it not finding vcredist dependencies.
 
-		Got an (internal) error in the following case (nivafjord pipe boundary model)
-			flux(bnd.water, layer.water[vert.specific]) ...
-
-			bnd.water.tox can be distributed over tox_index, but that dependency was ruled out due to lack of differing parametrization
-			layer.water.tox was distributed over tox_index with the actual dependency.
-			Caused problem when the dissolved flux were to be added to the connection aggregate (lack of index set).
-				Probably also a problem for vert.top etc.
-				It is a bit tricky for the framework to predetermine this.. But there is maybe something like this for graph aggregates already that could be applied here too.
-
 		Add forums on github page? (github Discussions)
 
 		Specific models
@@ -175,6 +168,15 @@
 
 	*** Intermediate-pri ***
 
+		Got an (internal) error in the following case (nivafjord pipe boundary model)
+			flux(bnd.water, layer.water[vert.specific]) ...
+
+			bnd.water.tox can be distributed over tox_index, but that dependency was ruled out due to lack of differing parametrization
+			layer.water.tox was distributed over tox_index with the actual dependency.
+			Caused problem when the dissolved flux were to be added to the connection aggregate (lack of index set).
+				Probably also a problem for vert.top etc.
+				It is a bit tricky for the framework to predetermine this.. But there is maybe something like this for graph aggregates already that could be applied here too.
+
 		If an input series is provided indexed over "subcatchment", but should be indexed over "water body" and "subcatchment" is a union member of "water body", that should be made to work. (and all similar cases) Just re-map the index.
 
 		Store name of expected model file in data sets (?)

docs/datafiledocs/new_project.md

@@ -55,7 +55,9 @@ layer : index_set("Layers") [ 35 ]
 
 The index set declaration only says what the indexes in the set are. Information about *connections* between them or *parameter* data attached to them is separately declared (see below).
 
-There are also the concepts of *sub-indexed* and *union* index sets as well as position maps, but this is more rare and will be documented later.
+There are also the concepts of [*sub-indexed* and *union* index sets](../mobius2docs/advanced_concepts.html#sub-indexed-index-sets), which is documented separately.
+
+The concept of position maps, will be documented later.
 
 ## Time series data
 

docs/mobius2docs/advanced_concepts.md

@@ -7,17 +7,130 @@ nav_order: 3
 
 # Advanced concepts
 
-This document is yet to be written. The guide is prioritized for now.
+This document is incomplete. The guide is prioritized for now.
 
 ## Index set distributions
 
-## Order of evaluation
+In a Mobius2 model, any compartment can be distributed over one or more index sets. This happens during declaration of the compartment in the model file.
+
+```python
+sc : compartment("Subcatchment")
+ly : compartment("Landscape units")
+
+air  : compartment("Atmosphere")
+soil : compartment("Soil", sc, lu)
+gw   : compartment("Groundwater, sc)
+```
+
+In the above example, there is only one global `air` compartment, while there is one `gw` box per subcatchment, and one `soil` box per pair of subcatchment and landscape unit (conceptually, each subcatchment is further-subdivided into landscape types).
+
+When a compartment is distributed, any state variable that has that compartment in its *location* can also be distributed over the given index sets. That means that the model can compute a separate value for that variable for each (tuple of) index(es) in those index set(s).
+
+The framework will try to determine if the value will actually be different, and if it can determine that the value will be the same across one index set, that index set will not be included in the distribution of the state variable.
+
+A state variable can pick up index set dependencies in a few different ways. The main one is from parameters. A parameter group can be distributed like one or more compartments. 
+
+```python
+par_group("Soil parameters", soil) {
+	fc : par_real("Field capacity", [m m], 100, 0, 500)
+	tc : par_real("Time constant", [day], 1, 1, 50)
+}
+```
+
+In the above example, the parameter group "Soil parameters" is distributed like `soil`. This means that it can be distributed over any of the index sets that `soil` is distributed over. What exact index sets are used is [determined in the data set](../datafiledocs/new_project.html#parameter-groups). For instance in this example, the user could distribute the "Soil parameters" over nothing (having the same conditions in all locations), or over landscape units (so that soils in the same landscape type are the same across subcatchments) or over both landscape type and subcatchment.
+
+Every parameter has the same distribution as its group. A state variable picks up the distributions of the parameters it accesses in its math expression (if it has one). This is because, if a parameter can be different for each index of an index set, any formula depending on that parameter must also be different across that index set. Input forcing series can similarly be distributed over index sets, and state variables that access an input forcing picks up these index sets.
+
+Every state variable also picks up index set dependencies from all other state variables they depend on. This can include
+* Access of the value of the other state variable in code.
+* Quantities depend on any fluxes affecting them.
+
+A state variable can only access parameters, input series and state variables from compartments that are distributed over a subset of the index sets of its own compartment. This rule is about the distribution of the compartment declarations, not the potentially smaller actual distributions of the state variables. This is because otherwise it would be ambiguous what index we are looking at in the excess index sets.
+
+For instance, in the above examples, any state variable in `soil` can access any state variable in `gw` and `air`, but not the other way around.
+
+There are some ways to get around this. For instance, you can access a variable from a higher distribution using [`aggregate`](math_format.html#special-directives). The aggregate sums the variable over the missing index sets, applying an [`aggregation_weight`](declaration_types.html#aggregation_weight) if one exists.
+
+In the example above, if `gw.water` accesses the value of `soil.temp`, it will get a weighted sum of the latter over the landscape unit index set.
+
+The aggregation is also automatically applied if a flux goes from a compartment with a higher number of index sets to a lower, for instance from `soil.water` to `gw.water`.
+
+Another way to access specific indexes of an index set is to use [location restrictions (below)](#location-restrictions).
+
+### Sub-indexed index sets
+
+Some times you may want make one index set have a different content depending on the index of another index set. For instance, a lagoon model may have a different amount of layers per basin.
+
+```python
+basin : index_set("Basin")
+layer : index_set("Layer") @sub(basin)
+```
+
+In this example, we say that the `layer` index set is *sub-indexed* to the `basin` index set, and that `basin` is the *parent* index set of `layer`. In the data set, you can now have a separate amount of layers per basin, declared using the [map format](declaration_format.html#maps), e.g.
+
+```python
+basin : index_set("Basin") [ "Inner fjord" "Outer fjord" ]
 
-## Grid connections
+layer : index_set("Layer") [!
+	"Inner fjord" : [ 20 ]
+	"Outer fjord" : [ 35 ]
+]
+```
 
-## Graph connections
+There are a few rules that must be followed to make this work. Any model entity that is distributed over a sub-indexed index set must also be indexed over the parent (otherwise you could not determine how many copies there should be). It is not allowed to have chains of sub-indexing, so a parent index set can't again be sub-indexed (the implementation complexity of this would be very high, and we haven't yet found a use case).
 
-## Location restrictions
+### Union index sets
+
+Some times it can be useful to be able to distribute an entity over a union of other index sets (not a tuple of them). For instance, you may want a separate state for the atmosphere (temperature, precipitation, ..) over lakes and subcatchments, but these are separate index sets. You can then make an index set that is a union of these.
+
+```python
+sc : index_set("Subcatchment")
+lk : index_set("Lake")
+wb : index_set("Water bodies") @union(sc, lk)
+```
+
+A union can have two or more members. If you now distribute `air` over `wb`, you can e.g. provide a separate input series of `air.precip` for each subcatchment and lake. Even though `soil` does not distribute over `wb` directly, state variables in `soil` can still access `air.precip` because `soil` is distributed over `sc`, and `sc` is a part of the `wb` union.
+
+For technical reasons,
+* You can't create unions of other index sets that are themselves unions.
+* A sub-indexed index set can't be a part of a union (but a parent index set can be).
+
+### Distributing quantities
+
+To not make the descriptions above too complicated up-front (and because it is less often used) we have omitted the fact that you can also distribute `quantity` declarations directly. This can be used to create several instances of the same type of substance, for instance different buoyancy classes of suspended particles, or different pollutants that use the same formulas (but are parametrized differently).
+
+```python
+sc : index_set("Subcatchment")
+pt : index_set("Pollutant type")
+
+river : compartment("Soil", sc)
+pollutant : quantity("Pollutant", pt)
+```
+
+In this example, "river.water.pollutant" can index over the tuple of `sc` and `pt` (one amount of pollutant per subcatchment and pollutant type). The amendments to distribution rules are pretty straightforward. For instance,
+* The allowed index set dependencies for a state variable includes the index sets coming from both its compartment and its quantities.
+* A parameter group can index like a quantity or a combination of compartments and quantities.
+* You can use `aggregate`, e.g. so that `river.water.tot_pollutant` can be computed using `aggregate(river.water.pollutant)`.
+
+## Connections
+
+Connections are used to link up different instances of the same compartment or multiple compartments, mainly for transport using fluxes.
+
+Without using connections, if you have a `soil` compartment and a `gw` compartment, you can have a flux from `soil.water` to `gw.water` implying that this flux happens within each subcatchment, but not across different subcatchments.
+
+On the other hand, if you want a flux (such as downstream discharge) between the river sections of two subcatchments you must use a `directed_graph` connection. There is also `grid1d` connections that position all the indexes of a given index set next to one another in a linear order.
+
+### Grid1D connections
+
+
+
+#### Location restrictions for grids
+
+### Directed graph connections
+
+#### Location restrictions for graphs
+
+## Order of evaluation
 
 ## Discrete fluxes
 

docs/mobius2docs/declaration_types.md

@@ -721,7 +721,7 @@ Optional notes:
 
 This creates an index set that compartments and quantities (and thus par_groups and state variables) can be distributed over.
 
-Sub-indexed and union index sets will be documented separately.
+[Sub-indexed and union index sets](advanced_concepts.html#sub-indexed-index-sets) are documented separately.
 
 ## connection
 
@@ -758,7 +758,7 @@ The specifics of how you can use this will be documented separately
 
 This arranges one or more indexed compartments along a directed graph. You can use it to path fluxes along networks of different compartments or different instances of these compartments.
 
-If you want to allow a single node to have multiple outgoing arrows, you need to provide an edge index set for the connection. The `edge_set` must have been declared as sub-indexed to the index_set of the node(s) that can have multiple outgoing edges, or potentially to a union of these if they are different.
+If you want to allow a single node to have multiple outgoing arrows, you need to provide an edge index set for the connection. The `edge_set` must have been declared as [sub-indexed](advanced_concepts.html#sub-indexed-index-sets) to the index_set of the node(s) that can have multiple outgoing edges, or potentially to a union of these if they are different.
 
 The regex body is currently not fully functional. It is supposed to describe how paths in the graph can look. For now just follow one of the below examples
 

docs/mobius2docs/math_format.md

@@ -114,7 +114,7 @@ An \<identifier\> is either the identifier of a local variable, the identifier o
 
 If you are in an expression with a *context location*, you can some times use shorthands for the location of a refereced state variable. For instance, if the context location is `river.water.oc`, and you try to access `temp`, if `temp` does not refer to a single value, Mobius2 will first look for `river.water.oc.temp`, then `river.water.temp` if the prior does not exist, and finally `river.temp`.
 
-These have the units and types they are declared with. If a value is indexed over index sets, it will primarily be accessed using the same indexes as the current expression are evaluated with. (This causes expressions to propagate index set dependencies to one another and to put some restrictions on what can be accessed. This will be separately documented).
+These have the units and types they are declared with. If a value is indexed over index sets, it will primarily be accessed using the same indexes as the current expression are evaluated with. (This causes expressions to propagate index set dependencies to one another and to put some restrictions on what can be accessed. See [separate documentation on distributions](advanced_concepts.html#index-set-distributions).
 
 Note that the value you get when you access a parameter value in the model is the one provided in the data set (corresponding to the current index combination). The default value in the parameter declaration is just a helper for somebody who create a new data set.
 
@@ -308,7 +308,7 @@ where a \<unit-declaration\> follows the [unit declaration format](units.html#th
 | `=>`     | Discards the unit of the lhs and replaces it with the rhs unit, keeping the same underlying numerical value. |
 | `=>>`    | Same as `=>`, but replaces the unit with the unit of the state variable of the outer function body similarly to `->>`. |
 
-To do the unit conversion, Mobius2 will durinc compilation generate a multiplication with a conversion factor (if it exists, otherwise it reports an error). For instance,
+To do the unit conversion, Mobius2 will during compilation generate a multiplication with a conversion factor (if it exists, otherwise it reports an error). For instance,
 
 ```python
 10[day] -> [s]

docs/mobiviewdocs/parameters.md

@@ -29,5 +29,5 @@ Edits to a parameter will apply immediately if you re-run the model, but are onl
 
 ## Special cases
 
-Some of the functionality can be a bit different if there are so-called "sub-indexed" index sets or if some parameters are on so-called "map" form. This is rare, and will be documented later.
+Some of the functionality can be a bit different if there are so-called [*sub-indexed*](../mobius2docs/advanced_concepts.html#sub-indexed-index-sets) index sets or if some parameters are on so-called "map" form. This is rare, and will be documented later.