[API Proposal]: Tensor Operators

[michaelgsharp]

Background and motivation

Tensors are often used to do multi-dimensional math in an efficient manner. We need to add these operators to our Tensor class to enable this behavior. The language team is working on adding 2 features to enable this behavior. First, the ability to restrict which types get certain operators, and second, the ability for users to implement their own compound operators, such as +=.

Note: this API is currently blocked waiting for 2 language features that are supposed to be implemented for .NET 10. All the following API are based on their current in progress work, but if anything major changes we will need to adjust accordingly.

API Proposal

namespace System.Numerics.Tensors;

public static partial class Tensor
{
    extension<TSelf, T>(TSelf)
        where T: INumber<T>
        where TSelf: IReadOnlyTensor<TSelf, T>
    {
        // All These will need to allocate
        public static IReadOnlyTensor<TSelf, T> operator *(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, T> operator *(TSelf tensor, TSelf tensor2) { ... }
        public static IReadOnlyTensor<TSelf, T> operator *(T scalar, TSelf tensor) { ... }

        public static IReadOnlyTensor<TSelf, T> operator +(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, T> operator +(TSelf tensor, TSelf tensor2) { ... }
        public static IReadOnlyTensor<TSelf, T> operator +(T scalar, TSelf tensor) { ... }

        // I removed some of the operators starting with a scalar. Should we have them anyways?
        public static IReadOnlyTensor<TSelf, T> operator %(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, T> operator %(TSelf tensor, TSelf tensor2) { ... }

        public static IReadOnlyTensor<TSelf, T> operator /(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, T> operator /(TSelf tensor, TSelf tensor2) { ... }

        public static IReadOnlyTensor<TSelf, T> operator -(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, T> operator -(TSelf tensor, TSelf tensor2) { ... }


       // Is it ok to have = not return true/false for all the following comparisons
        public static IReadOnlyTensor<TSelf, bool> operator ==(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator ==(TSelf tensor, TSelf tensor2) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator ==(T scalar, TSelf tensor) { ... }

        public static IReadOnlyTensor<TSelf, bool> operator !=(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator !=(TSelf tensor, TSelf tensor2) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator !=(T scalar, TSelf tensor) { ... }

        public static IReadOnlyTensor<TSelf, bool> operator <(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator <(TSelf tensor, TSelf tensor2) { ... }

        public static IReadOnlyTensor<TSelf, bool> operator >(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator >(TSelf tensor, TSelf tensor2) { ... }

        public static IReadOnlyTensor<TSelf, bool> operator <=(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator <=(TSelf tensor, TSelf tensor2) { ... }

        public static IReadOnlyTensor<TSelf, bool> operator >=(TSelf tensor, T scalar) { ... }
        public static IReadOnlyTensor<TSelf, bool> operator >=(TSelf tensor, TSelf tensor2) { ... }
    }
}

    extension<TSelf, T>(TSelf)
        where T: INumber<T>
        where TSelf: ITensor<TSelf, T>
{
    extension<TSelf, T>(ITensor<TSelf, T>) where T: INumber<T>
    {
        // In place updates need no allocation
        public static void operator *=(this TSelf tensor, T scalar) { ... }
        public static void operator *=(this TSelf tensor, TSelf tensor2) { ... }

        public static void operator +=(this TSelf tensor, TSelf tensor2) { ... }
        public static void operator +=(this TSelf tensor, T scalar) { ... }

        public static void operator %=(this TSelf tensor, T scalar) { ... }
        public static void operator %=(this TSelf tensor, TSelf tensor2) { ... }

        public static void operator /=(this TSelf tensor, T scalar) { ... }
        public static void operator /=(this TSelf tensor, TSelf tensor2) { ... }

        public static void operator -=(this TSelf tensor, T scalar) { ... }
        public static void operator -=(this TSelf tensor, TSelf tensor2) { ... }
    }
}

API Usage

Tensor<int> tensor = Tensor.Create<int>([1, 2, 3, 4], [2, 2]);

// After this the values inside should be [3, 4, 5, 6]
tensor += 2;

// after this the values inside should [9, 12, 15, 18] 
tensor *= 3;

// Allocates a new tensor with values [10, 13, 16, 19]
Tensor<int> tensor2 = tensor + 1;

// Each element will be true/false base on if it equals the element in the same location in the other tensor.
Tensor<bool> equalTensor = tensor == tensor2;

Alternative Designs

None. We needed this functionality and so the language team figured out the best way to incorporate it in C# and we are just adopting it.

Risks

Low risk because it's a preview type and its all new features. The main risk will be around the language team and any delays they may have.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

[tannergooding]

@michaelgsharp, some notes:

• The extension operators would be defined on the Tensor static class, not on the interfaces
• They should likely be taking TSelf rather than IReadOnlyTensor<TSelf, T>, to avoid boxing for value types implementing the interfaces
• The compound assignment operators would be void returning and take this TSelf as the first parameter.
• We should potentially also consider if we need both this TSelf and ref this TSelf for the value type variants (which should be possible with the new extension feature)
• We do need the set of TSelf op TSelf, T op TSelf and TSelf op T where that's relevant (same places we exposed the set for the named methods like Add, Multiply, or Divide).

[tannergooding]

There's some other operator extensions that look to be missing as well. Ones like &, |, ^, ~, unary -, unary +, ++, --, <<, >>, and >>>.

[michaelgsharp]

I wasn't sure if the other operators really applied to the tensors. but I'll get them added in along with your other suggestions.