[WIP] Create metric and billing interface outline #173

lasley · 2016-12-01T03:57:14Z

This is a prelim outline of what I have been describing in #157. Nothing functional at the moment other than data schemas and a few theoretical methods.

There's also a shell of a metric interface definition, but that was just so I had something to connect the billing models to. I'm still coming up with a real plan there.

lasley · 2016-12-01T04:01:20Z

sale_clouder/models/product_template.py

+    invoice_policy = fields.Selection(
+        selection_add=[
+            ('threshold', 'Invoice and Enforce a Threshold'),
+            ('usage', 'Invoice Based on Usage'),


This may be removed in favor of using pre-existing delivery policy, which represents the same thing for the most part

lasley · 2016-12-01T19:37:46Z

clouder_metric/__init__.py

@@ -0,0 +1,5 @@
+# -*- coding: utf-8 -*-
+# Copyright 2016 LasLabs Inc.
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).


Change all to LGPL

YannickB · 2016-12-01T20:44:15Z

Great @lasley , I had a first look at it but I'll have a second one tomorrow. Thanks!

YannickB

Two comment, but other than that LGTM

YannickB · 2016-12-02T09:11:36Z

sale_clouder/models/clouder_contract_line.py

+        required=True,
+        ondelete='restrict',
+    )
+    service_id = fields.Many2one(


Don't forget that we shall also be able to invoice a base, not only a service.

Ah good call, I knew it seemed like I was missing a layer of abstraction!

Just so I'm clear, what metrics are going to be monitored on base? That's just going to be flat rate for the most part right?

Hum the main one I see we can use for billing is the number of users

Hmmmm but wouldn't the number of users be dictated in the Service itself? The base represents a collection of services doesn't it?

Nope it's the other way around.

A service is a set of containers, aka one Odoo instance.
A base represent both a database and an url. You can have several database in one service, like the Odoo SA offer.

And you want to count the number of user for each database of course.

YannickB · 2016-12-02T09:14:48Z

clouder_metric/models/clouder_metric_value.py

+from odoo import api, fields, models
+
+
+class ClouderMetricValue(models.Model):


I don't really understand the use of this model, because we said the value would be recovered realtime from ELK. If we need to store it here, we shall probably have a datetime field to know when the value was recovered.

That said, I understand this is a POC and it will probably change when we get close to the end.

Yeah everything is going to be stored in ELK, but we need some way to store the aggregate billing metrics so they don't change out from under us after they have been used for billing.

Yup you're totally right about the need for datetimes indicating the aggregation interval. I added that in my RFC at https://github.com/clouder-community/clouder/issues/174

This module is likely going to be removed from this PR, it was just here so I could connect accounting to something while figuring that all out.

tedsalmon · 2017-03-02T05:18:44Z

sale_clouder/models/clouder_contract.py

+
+    @api.multi
+    def _get_quantity_usage(self, account_line, invoice):
+        """ It provides a quantity based on unbilled and used metrics """


Hey @lasley,

My assumption is that we need to get a RecordSet of clouder.metric.interface to return here, right? I'm just failing to find a good relation between the two. Perhaps it just wasn't implemented yet?

Ummm no, it would be an actual numeric. All of these _get_quantity_* methods would be used to determine the product_qty of the invoice line item.

Take a look at _get_quantity_flat

codecov-io · 2017-03-10T23:26:04Z

Codecov Report

Merging #173 into master will increase coverage by 1.37%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #173      +/-   ##
==========================================
+ Coverage   31.53%   32.91%   +1.37%     
==========================================
  Files          73       79       +6     
  Lines        5755     5873     +118     
==========================================
+ Hits         1815     1933     +118     
  Misses       3940     3940

Impacted Files	Coverage Δ
sale_clouder/models/clouder_contract_line.py	`100% <100%> (ø)`
clouder_metric/models/clouder_metric_interface.py	`100% <100%> (ø)`
sale_clouder/models/clouder_contract.py	`100% <100%> (ø)`
sale_clouder/models/product_template.py	`100% <100%> (ø)`
clouder_metric/models/clouder_metric_type.py	`100% <100%> (ø)`
clouder_metric/models/clouder_metric_value.py	`100% <100%> (ø)`
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b275e75...400f151. Read the comment docs.

tedsalmon · 2017-03-11T00:15:03Z

Hey guys,

This is should be ready for review. Let me know if I missed anything.

Thanks!
-Ted

lasley · 2017-03-13T17:40:36Z

clouder_metric/__manifest__.py

@@ -0,0 +1,23 @@
+# -*- coding: utf-8 -*-
+# Copyright 2016 LasLabs Inc.
+# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).


This has to be AGPL due to contract_variable_quantity depends - assuming we're still using logic from it.

Argh... This may be a problem, the metric module will probably be deployed in all infrastructures. If it becomes AGPL, we can practically assume that all Clouder will become AGPL, this is not the same case than clouder_asynchronous.

I'm afraid I have to be against such move just because we use this dependency. Either we can convince people that contract_variable_quantity is considered a component and shall be LGPL, or we'll have to do without it.

That being said, I think I'll not block the PR just for that, we can find a solution later and merge it AGPL for the time being.

@YannickB - I think we'll be fine here. Reason being is that the metrics and sales modules are far from the core, nothing will need to depend on them, and there will likely not be third party proprietary changes.

Anything to do with contract is licensed AGPL, primarily due to it being ripped out from v8 - before the LGPL license switch in v9. I don't really want to rewrite the contract module at this time, but that would be the requirement to move from this.

Very well, I'm fine anyway we'll figure out what happen later.

lasley · 2017-03-13T17:41:28Z

clouder_metric/models/clouder_metric_interface.py

+    This object receives all attributes from ``clouder.metric.type``.
+    """
+
+    _name = 'clouder.metric.interface'


_description key on all new models

lasley · 2017-03-13T17:42:00Z

clouder_metric/models/clouder_metric_type.py

+class ClouderMetricType(models.Model):
+    """ It provides context for usage metric types """
+
+    _name = 'clouder.metric.type'


_description key on all new models

lasley · 2017-03-13T17:43:06Z

clouder_metric/models/clouder_metric_type.py

+        return _("# Python code. \n"
+                 "Use `value = my_value` to specify the final calculated "
+                 " metric value. This is required. \n"
+                 "Optionally use ``uom == product_uom_record`` to change the "


lasley · 2017-03-13T17:43:33Z

clouder_metric/models/clouder_metric_type.py

+                 "units that the metric is being measured in. \n"
+                 "# You can use the following variables: \n"
+                 "#  - self: browse_record of the current ID Category "
+                 "browse_record. \n"


Double browse_record

lasley · 2017-03-13T18:02:52Z

sale_clouder/models/clouder_contract.py

+                line being referenced.
+            invoice (account.invoice): Invoice that is being created.
+        Returns:
+            (float) Quantity with no calculations performed


lasley · 2017-03-13T18:03:20Z

sale_clouder/models/clouder_contract.py

+                line being referenced.
+            invoice (account.invoice): Invoice that is being created.
+        Returns:
+            (float) Quantity to use on invoice line in the UOM defined on the


lasley · 2017-03-13T18:04:21Z

sale_clouder/models/clouder_contract.py

+                return False
+            start = fields.Datetime.from_string(rec.date_start)
+            end = fields.Datetime.from_string(rec.date_end)
+            return start >= start_date and end <= inv_date


Add blank line

lasley · 2017-03-13T18:05:03Z

sale_clouder/models/clouder_contract.py

+            end = fields.Datetime.from_string(rec.date_end)
+            return start >= start_date and end <= inv_date
+        usage_values = vals.filtered(filter)
+        for val in usage_values:


usage = sum(usage_values.mapped('value'))

lasley · 2017-03-13T18:05:15Z

sale_clouder/models/clouder_contract_line.py

+class ClouderContractLine(models.Model):
+    """ It provides the link between billing and Clouder Services. """
+
+    _name = 'clouder.contract.line'


_description

* Fix docstrings * Implement unimplemented methods * Fix bugs found * Add test coverage * Add README [IMP] clouder_metric: Finish module * Fix/Add docstrings * Add test coverage * Fix logic * Add README

* Update license to AGPL * Add model `_description` * Clean up lint issues

* Update license to AGPL * Add model `_description` * Clean up lint issues * Improve usage metric aggregation logic

tedsalmon · 2017-03-13T22:07:49Z

Thanks @lasley! I've attended to your comments in commit 8c7645c & 400f151

Again, I was unable to switch the fields.Selection field to fields.Reference because the latter is a reference to a specific record of a model; not the model itself.

I also boiled down some of the logic used to calculate the usage quantity in clouder.contract

Thanks!

YannickB · 2017-03-14T12:07:20Z

Hi guys,

Good job ! I'll make my review this afternoon, hope we can merge it quickly after that. Thanks again for your hard work.

YannickB

Ok, except one minor comment, I'm ok for this PR.

Note I've yet to make a functional review. I think make optimistic merge and correct functional issues through next PRs is the way to go here, so I'm ready to merge when @lasley approve.

YannickB · 2017-03-14T13:35:34Z

sale_clouder/__manifest__.py

@@ -0,0 +1,25 @@
+# -*- coding: utf-8 -*-


Please make a symbolic link from openerp.py to manifest.py to ensure the module can be installed on Odoo 9 (see my last commit on master)

YannickB · 2017-03-14T13:35:46Z

clouder_metric/__manifest__.py

@@ -0,0 +1,23 @@
+# -*- coding: utf-8 -*-


Please make a symbolic link from openerp.py to manifest.py to ensure the module can be installed on Odoo 9 (see my last commit on master)

I'm not so sure about this. What about the changed namespace (from odoo vs from openerp)? I'm pretty sure I have also used some new API decorators that will not work on v9 (api.model_cr_context)

Well at the very least the tricks worked on my dev instance, and all other modules now have the symbolic link. Anyway I'll not block the PR for this, so I'm just waiting for your approval.

lasley · 2017-03-14T21:29:19Z

LGTM thanks @tedsalmon. We'll likely need to make some changes when we actually implement with the Elastic connector, but this is a great start for a merge IMO.

YannickB · 2017-03-14T21:45:23Z

OK, LGTM for me too then. I'm merging, thanks @tedsalmon @lasley for the hard work !

lasley commented Dec 1, 2016

View reviewed changes

lasley force-pushed the feature/master/usage-and-billing branch 2 times, most recently from e88d57f to b0bff52 Compare December 1, 2016 04:03

lasley commented Dec 1, 2016

View reviewed changes

YannickB reviewed Dec 2, 2016

View reviewed changes

lasley force-pushed the feature/master/usage-and-billing branch 2 times, most recently from bca9547 to 3051df9 Compare December 20, 2016 19:49

tedsalmon reviewed Mar 2, 2017

View reviewed changes

tedsalmon force-pushed the feature/master/usage-and-billing branch from 3f9e1ff to 41717e5 Compare March 9, 2017 02:06

tedsalmon force-pushed the feature/master/usage-and-billing branch 2 times, most recently from bbfdc58 to 92cafec Compare March 11, 2017 00:05

tedsalmon force-pushed the feature/master/usage-and-billing branch from 92cafec to d57d815 Compare March 13, 2017 17:32

lasley commented Mar 13, 2017

View reviewed changes

lasley and others added 4 commits March 13, 2017 14:04

Create metric and billing interface outline

11b11f5

Playing around with data structures

5437146

[IMP] sale_clouder: Finish module

e3a0b89

* Fix docstrings * Implement unimplemented methods * Fix bugs found * Add test coverage * Add README [IMP] clouder_metric: Finish module * Fix/Add docstrings * Add test coverage * Fix logic * Add README

[IMP] clouder_metric: Fixes per PR

8c7645c

* Update license to AGPL * Add model `_description` * Clean up lint issues

tedsalmon force-pushed the feature/master/usage-and-billing branch from 489375c to 8c7645c Compare March 13, 2017 21:05

[IMP] sale_clouder: Fixes per PR

400f151

* Update license to AGPL * Add model `_description` * Clean up lint issues * Improve usage metric aggregation logic

tedsalmon force-pushed the feature/master/usage-and-billing branch from 297f6c7 to 400f151 Compare March 13, 2017 21:45

YannickB suggested changes Mar 14, 2017

View reviewed changes

YannickB approved these changes Mar 14, 2017

View reviewed changes

YannickB merged commit 59dd6ea into YannickB:master Mar 14, 2017

		from odoo import api, fields, models


		class ClouderMetricValue(models.Model):

[WIP] Create metric and billing interface outline #173

[WIP] Create metric and billing interface outline #173

Conversation

lasley commented Dec 1, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YannickB commented Dec 1, 2016

YannickB left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Mar 10, 2017 • edited Loading

Codecov Report

tedsalmon commented Mar 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tedsalmon commented Mar 13, 2017

YannickB commented Mar 14, 2017

YannickB left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lasley commented Mar 14, 2017

YannickB commented Mar 14, 2017

lasley commented Dec 1, 2016 •

edited

Loading

codecov-io commented Mar 10, 2017 •

edited

Loading