How to optimize simple memory stream for performance

[artemiusgreat]

Is there any documentation explaining how to minimize timeouts and increase responsiveness of Orleans streams?

Default parameters of simple memory stream are not optimal. If I create a timer executed every 1 microsecond and send this data to a stream, then stop the timer, stream still keeps sending data for couple of seconds, meaning that due to limited throughput the messages in the stream are piling up and even when streaming stops, it takes some time for the stream to send all accumulated messages.

One of the options to match throughput of the consumer (client) and producer (stream) is to add simple flow control when producer subscribes to its own stream and sends new message only when the previous one is received. This will ensure that messages are not piling up but I would like to increase responsiveness of the stream instead of blocking it.

public class SomeGrain 
{
   int message = 0;
   IAsyncStream<int> messageStream;

  public override async Task OnActivateAsync(CancellationToken cleaner)
  {
    messageStream = this.GetStreamProvider(string.Empty).GetStream<int>(string.Empty, Guid.Empty);
    await messageStream.SubscribeAsync((o, x) => message = o);
  }

  public Task Start() 
  {
      var nextMessage = 1;
      var counter = new Timer(TimeSpan.FromMicroseconds(1));

      counter.Enabled = true;
      counter.Elapsed += async (sender, e) => 
      {
        if (Equals(nextMessage, message) is false) await messageStream.OnNextAsync(nextMessage++);
      };
  }
}

[SebastianStehle]

I would not mix up latency and hroughput. e.g. in kafka messages are basically polled. You usually want to have a small wait time so that you do not pull too often if there is nothing in the queue. But when there are messages you can make the assumption that there would be probably more and skip the delay.

Are you sure, that you do not want to have a simple request-response model?

[artemiusgreat]

Thank you for the reply. To some extent what you're saying makes sense and similar approach is shown in one of the sample apps.

The thing is that when you integrate some trading brokerage API, you need to subscribe to price notifications sent via sockets. Broker-producer sends this notification and consumer must process it, e.g. make trading decision based on it, so streaming / broadcasting approach aligns here better than polling.

[artemiusgreat]

Streams don't seem to be suitable for real-time low-latency messages, so I switched to SignalR.

[ReubenBond]

I agree with that, Streams are primarily intended to be durable and high throughput, not minimal latency. That being said, there is also IAsyncEnumerable, observers, and Broadcast Channel (used to be called SMS streams - they're point to point and unbuffered streams) these are all low latency options.