User defined compound assignment operators

[AlekseyTs]

Champion issue: #9101

Allow user types to customize behavior of compound assignment operators in a way that the target of the assignment is
modified in-place.

Motivation

C# provides support for the developer overloading operator implementations for user-defined type.
Additionally, it provides support for "compound assignment operators" which allow the user to write
code similarly to x += y rather than x = x + y. However, the language does not currently allow
for the developer to overload these compound assignment operators and while the default behavior
does the right thing, especially as it pertains to immutable value types, it is not always
"optimal".

Given the following example

class C1
{
    static void Main()
    {
        var c1 = new C1();
        c1 += 1;
        System.Console.Write(c1);
    }
    
    public static C1 operator+(C1 x, int y) => new C1();
}

with the current language rules, compound assignment operator c1 += 1 invokes user defined + operator
and then assigns its return value to the local variable c1. Note that operator implementation must allocate
and return a new instance of C1, while, from the consumer's perspective, an in-place change to the original
instance of C1 instead would work as good (it is not used after the assignment), with an additional benefit of
avoiding an extra allocation.

When a program utilizes a compound assignment operation, the most common effect is that the original value is
"lost" and is no longer available to the program. With types which have large data (such as BigInteger, Tensors, etc.)
the cost of producing a net new destination, iterating, and copying the memory tends to be fairly expensive.
An in-place mutation would allow skipping this expense in many cases, which can provide significant improvements
to such scenarios.

Therefore, it may be beneficial for C# to allow user types to
customize behavior of compound assignment operators and optimize scenarios that would otherwise need to allocate
and copy.

[huoyaoyuan]

For functional and immutability perspective, how to make sure the instance isn't shared inappropriately? Like:

BigInteger a = 1;
BigInteger b = a;
a += 1;

Should this be equivalent to the following?

BigInteger a = 1;
BigInteger b = a;
a = a + 1;

It may also cause concern about thread safety etc. In other words, reusing storages in compound assignment are only safe if unique ownership is provided.

Although this doesn't affect the operator support in C# itself, it can affect API designs leveraging this pattern. Can we provide more?

[tannergooding]

Such a thing is already problematic for instance methods on value types

A struct can contain indirect state and so copying/mutating can result in changes that are visible outside that specific struct instance.

+= would be taking ref T this and so would be free to do what is appropriate for that type based on whether its fields were mutable or readonly.

Even readonly instance methods don't guarantee that the state is immutable, it simply guarantees that it cannot be mutated via this. Consider for example the following code which is completely safe, legal, and valid:

using System;

// Prints:
//    1
//    2
MyStruct tmp = new MyStruct(1);
tmp.M(ref tmp);

readonly struct MyStruct
{
    private readonly int _x;

    public MyStruct(int x)
    {
        _x = x;
    }

    public void M(ref MyStruct m)
    {
        Console.WriteLine(_x);
        m = new MyStruct(2);
        Console.WriteLine(_x);
   }
}

[tannergooding]

That is, this isn't really exposing anything "new" that couldn't be done before with regards to say (pseudo-code):

BigInteger a = 1;
BigInteger b = a;
a.Add(1);
Console.WriteLine(b);

It is ultimately up to the implementor of the API to determine what semantics are appropriate. To determine what state should be aliased vs what state should be kept to itself and therefore if the above should print 1 or 2

[hamarb123]

I was excited to see this feature proposal :)

Should readonly modifier be allowed in structures?
It feels like there would be no benefit in allowing to mark a method with readonly when the whole purpose of the method is to modify the instance.

Re this: allowing readonly members can be quite sensible in the face of types with move-only semantics (just without built-in compiler errors/warnings; but these could be provided by an analyser even today, or we might get proper c# support for move-only types some day if we're lucky, etc.), or other types with aliasing semantics as mentioned by tanner in this comment (although, for the majority of the cases where you'd have aliasing semantics, I suspect you'd also want move-only semantics); specifically, if the struct is backed by some sort of reference, then it can mutate while still being readonly (e.g., you could have a type backed by some already-allocated pooled memory, that has move-only semantics). Even though this would not be the common use case of stuff like +=, it would be good to support it as there are legitimate use cases for marking them readonly imo - the scenarios where it might be sensible to mark them readonly is quite similar to the scenarios where you might mark a set accessor readonly. The consideration is that readonly doesn't necessarily mean it doesn't mutate what the instance represents, it more means it doesn't mutate memory directly within the struct (these are 2 separate things).

Also, another question is how it works in readonly structs - it'd probably be good to still be able to define these on readonly structs, but their implementations (for the most part) probably need to be mutating. Some thought might need to be given to this - perhaps it's like init members where they're mutating anyway (at least by default, it would be good if you could still mark them readonly imo).

[jnm2]

If the struct is readonly and it applies its operations to an array field for instance, why block in-place modifications on this struct?

[jilles-sg]

As noted in another answer, this proposal does not add much (if anything) for the existing kinds of types. However, for move-only types it does add new capabilities. If you have a move-only type behind a ref or in a field of a reference type, you can take a ref to it but you cannot move out of it (the compiler cannot prevent access to the storage, so it must remain valid). So the proposed x += y would work but x = x + y would not work (especially if y is instead a method call or even an await).

The workaround with the existing language is to use a method which can take x by reference.

A subtle side effect of the proposal will be that the order of evaluation will be different for types that adopt user-defined compound assignment operators. Normally, x += y reads x before evaluating y, but a method call taking x by reference will evaluate y before reading x. For a new language the new order would be better (it would also improve += await), but the change may be rather confusing.

A contrived example:

int x = 1;
x += f();
Console.WriteLine($"x = {x}");

int f() {
    x = 100;
    return 4;
}

This prints x = 5; the assignment to x within f is lost.

[tannergooding]

but a method call taking x by reference will evaluate y before reading x

That's not how that works. You can see for the following:

public static bool M(int x, Func<int> y) => x.Equals(y());

The following IL generated is:

        IL_0000: ldarga.s x
        IL_0002: ldarg.1
        IL_0003: callvirt instance !0 class [System.Runtime]System.Func`1<int32>::Invoke()
        IL_0008: call instance bool [System.Runtime]System.Int32::Equals(int32)
        IL_000d: ret

This clearly shows that the address of x is evaluated prior to evaluating y.

The same shows for something like: bool M(Func<int> x, Func<int> y) => x().Equals(y()); where M(() => { Console.Write("X"); return 1; }, () => { Console.Write("Y"); return 1; }); would print XY

or for the following, which prints "XY1":

C.X() += Y();
Console.WriteLine(C.Value);

int Y()
{
    Console.Write("Y");
    return 1;
}

class C
{
    public static int Value = 0;

    public static ref int X()
    {
        Console.Write("X");
        return ref Value;
    }
}

The instance method evaluates this, then the parameters in left to right order. So x += y will also do the same, evaluating x and if necessary placing it in a temporary, then evaluating y, then calling op_AdditionAssignment and behave identically

[colejohnson66]

I think the current behavior because the existing lowering turns x += f(); into x = x + f();. Due to C#'s evaluation rules, x is evaluated (i.e., pushed to the stack) before f(). Correct me if I'm wrong, but this change would modify x += f() to lower as x.op_AdditionAssignment(x /* this */, f());, which would give the same result (evaluating x first).

[hamarb123]

@jilles-sg x += f(); does not lower it into x = x + f() literally, but rather with x only being evaluated once with (and before f()), you can see the IL generated for the below today:

using System;
public class C {
    public void M() {
        A() += B();
    }
    public ref int A()
    {
        throw null!;
    }
    public int B() => 0;
}

Link

So, the change to evaluate x & then f() & then call op_AdditionAssignment will not change the evaluation order of anything.

[jilles-sg]

Correct me if I'm wrong, but this change would modify x += f() to lower as x.op_AdditionAssignment(x /* this */, f());, which would give the same result (evaluating x first).

I'm looking at https://github.com/dotnet/csharplang/blob/4d06e0bb1f7e16b3f0fab2b379dc87099e33af5c/proposals/user-defined-compound-assignment.md which lowers it as x.op_AdditionAssignment(f());. So if x is a value type, only a reference to it is passed to op_AdditionAssignment and the value of x will only be read in the op_AdditionAssignment implementation, which necessarily runs after f has been called.

[hamarb123]

As tanner pointed out to me on the c# discord when I figured out what you meant, you can already get this behaviour by taking the left parameter (or both) by in using the existing operator + & get this behaviour. This indicates to me that it's probably already UB to rely on it either way (& the c# spec didn't seem to spell out which was expected when I looked at part of it).

[jnm2]

The spec proposes that when there is an overload resolution failure among declared compound operators (e.g. +=), there is no fallback to the non-compound operator (e.g. +) as there would be if the compound operators were not declared.

This makes it a breaking change to add any compound assignment operator to a type in the following scenario:

var x = new MyType();
x += 4; // Breaks if the `+=(MyType, MyType)` operator below is added, and the `+=(MyType, int)` operator is overlooked.

class MyType
{
    public void operator +=(MyType right) => ...

    public static MyType operator +(MyType left, MyType right) => ...
    public static MyType operator +(MyType left, int right) => ...
    // Irrelevant to the example, but often declared:
    public static MyType operator +(int left, MyType right) => ...
}

For safety, if any particular compound operator is declared, I propose a compiler warning if it is missing any overloads that are present among its matching non-compound operator declarations. (Non-compound operator overloads where the first operand is a different type can't have a compound operator counterpart, and are ignored for this check.)

[AlekseyTs]

This doesn't look like an accurate interpretation of the spec. The added operator won't be applicable, therefore, it won't prevent use of a +. Fall back won't happen only in presence of applicable to the arguments instance operators.