feat: add `download` and `flash` gRPC commands #50

MatteoPologruto · 2025-12-16T11:00:50Z

Motivation

Add grpc support for download and flash commands.

Change description

Add grpc support for download and flash commands. The flash progress coming from qdl has not been implemented yet. This is an example of the command's output:

{
  "downloadProgress": {
    "update": {
      "downloaded": "2357118536",
      "totalSize": "2357118536"
    }
  }
}
{
  "downloadProgress": {
    "end": {
      "success": true
    }
  }
}
{
  "extractionProgress": {
    "name": "extract",
    "message": "Start"
  }
}
{
  "extractionProgress": {
    "name": "extract",
    "message": "arduino-unoq-debian-image-20251127-441/disk-sdcard.img.root"
  }
}
{
  "extractionProgress": {
    "name": "extract",
    "completed": true
  }
}
{
  "flashProgress": {
    "name": "flash",
    "message": "Starting"
  }
}
{
  "flashProgress": {
    "name": "flash",
    "completed": true
  }
}

Additional Notes

$ grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/flasher/v1/commands.proto \
  -d '{"version": "latest", "temp_path": "<path>", "preserve_user": false}' \
  127.0.0.1:50052 \
  cc.arduino.flasher.v1.Flasher.Flash

Reviewer checklist

PR addresses a single concern.
PR title and description are properly filled.
Changes will be merged in main.
Changes are covered by tests.
Logging is meaningful in case of troubleshooting.

service/service_stream_utility.go

lucarin91 · 2025-12-19T13:41:09Z

rpc/cc/arduino/flasher/v1/common.proto

+message TaskProgress {
+  // Description of the task.
+  string name = 1;
+  // Additional information about the task.
+  string message = 2;
+  // Whether the task is complete.
+  bool completed = 3;
+  // Amount in percent of the task completion (optional).
+  float percent = 4;


I think we should add a better object for the extract and flash command

lucarin91 · 2025-12-19T13:44:38Z

rpc/cc/arduino/flasher/v1/commands.proto


 service Flasher {
    rpc List(ListRequest) returns (ListResponse) {};
+    rpc Download(DownloadRequest) returns (stream DownloadResponse);


Why is there a Download grpc command? Didn't we agree on exposing only the flash one?

Yes we did, but it comes for free since it shares most of the logic with the Flash command. I would keep it, even if it is not needed right now

It's required to implement the CLI download command

We aren't implementing the CLI download command but the gRPC interface, so I don't want to include stuff that will not be used. Can we focus on the things we need?

lucarin91 · 2025-12-19T13:47:12Z

rpc/cc/arduino/flasher/v1/commands.proto

+    DownloadProgress download_progress = 1;
+    // Progress of the extraction.
+    TaskProgress extraction_progress = 2;
+    // Progress of qdl flashing.
+    TaskProgress flash_progress = 3;
+    // Download result.
+    Result result = 4;


Here, I would expect to have something like

Suggested change

DownloadProgress download_progress = 1;

// Progress of the extraction.

TaskProgress extraction_progress = 2;

// Progress of qdl flashing.

TaskProgress flash_progress = 3;

// Download result.

Result result = 4;

DownloadProgress download_progress = 1;

ExtractProgress extraction_progress = 2;

FlashProgress flash_progress = 3;

Result result = 4;

each of which has its own internal states

rpc/cc/arduino/flasher/v1/commands.proto

cmaglie · 2025-12-19T12:54:26Z

rpc/cc/arduino/flasher/v1/commands.proto

+  // The path in which the image will be downloaded and extracted.
+  string temp_path = 2;


Suggested change

// The path in which the image will be downloaded and extracted.

string temp_path = 2;

// The path in which the image will be downloaded and extracted.

// If an image file is already present, it will not be downloaded again.

string temp_path = 2;

This is actually a bug because flashing fails if the file is already present. With the CLI command we override the files each time, so we should probably do the same gRPC side or preemptively delete any possible leftovers.
We could also go for a recovery or skip-download approach.

rpc/cc/arduino/flasher/v1/commands.proto

cmaglie · 2025-12-19T13:33:01Z

rpc/cc/arduino/flasher/v1/commands.proto

+
+message FlashResponse {
+  message Result {
+    // Empty message, reserved for future expansion.


Should we return the full path to the downloaded image file?

Probably not because at that point the image would have already been flashed onto the board and deleted locally. I wonder if it's actually needed, I don't see how it could be useful outside of providing a Flashing succeeded message.

service/service_flash.go

cmaglie · 2025-12-19T13:45:40Z

service/service_flash.go

+	// Check if there is enough free disk space before downloading and extracting an image
+	d, err := disk.Usage(req.TempPath)
+	if err != nil {
+		return err
+	}
+	if d.Free/updater.GiB < updater.DownloadDiskSpace {
+		return fmt.Errorf("download and extraction requires up to %d GiB of free space", updater.DownloadDiskSpace)
+	}
+
+	client := updater.NewClient()
+	manifest, err := client.GetInfoManifest(ctx)
+	if err != nil {
+		return err
+	}
+
+	var rel *updater.Release
+	if req.Version == "latest" || req.Version == manifest.Latest.Version {
+		rel = &manifest.Latest
+	} else {
+		for _, r := range manifest.Releases {
+			if req.Version == r.Version {
+				rel = &r
+				break
+			}
+		}
+	}
+
+	if rel == nil {
+		return fmt.Errorf("could not find Debian image %s", req.Version)
+	}
+
+	tmpZip := paths.New(req.GetTempPath(), "arduino-unoq-debian-image-"+rel.Version+".tar.zst")
+	defer func() { _ = tmpZip.RemoveAll() }()
+
+	if err := updater.DownloadFile(ctx, tmpZip, rel, downloadCB, downloader.Config{}); err != nil {
+		return err
+	}


This is duplicated code with the Download service. Please, move the duplicated code in a private function that is called from both Download and Flash services.

Moreover, in the Download service, the check for free space is not performed. This will be solved automatically when the duplicated code is factored out.

service/service_flash.go

Co-authored-by: Luca Rinaldi <lucarin@protonmail.com>

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

MatteoPologruto self-assigned this Dec 16, 2025

MatteoPologruto added the enhancement New feature or request label Dec 16, 2025

MatteoPologruto force-pushed the download-grpc branch from a7a403d to 9aaa1d0 Compare December 16, 2025 13:53

MatteoPologruto changed the title ~~feat: add download grpc command~~ feat: add download and flash gRPC commands Dec 16, 2025

MatteoPologruto force-pushed the daemon-grpc branch from fc54f77 to a05d1d9 Compare December 16, 2025 14:01

MatteoPologruto force-pushed the download-grpc branch from 9aaa1d0 to 6278ad8 Compare December 16, 2025 14:02

MatteoPologruto linked an issue Dec 16, 2025 that may be closed by this pull request

[App Lab integration] Expose download and extract #47

Open

Base automatically changed from daemon-grpc to main December 18, 2025 14:02

MatteoPologruto added 7 commits December 18, 2025 15:04

First implementation of the grpc download function

74915e6

Add stream utility functions

6391fc0

Add grpc download function

51e0141

nit: re-order some imports

97f1fd5

feat: add grpc flash command

682ace3

Delete files if an error occurs during download

67a465a

Run go fmt

7c5b688

MatteoPologruto force-pushed the download-grpc branch from 6e387d9 to 7c5b688 Compare December 18, 2025 14:05

MatteoPologruto added 2 commits December 18, 2025 15:29

Appease linter

5f3bc99

Check the downloaded file's hash

e4cbc44

MatteoPologruto marked this pull request as ready for review December 19, 2025 11:27

MatteoPologruto requested review from a team and cmaglie December 19, 2025 11:28

lucarin91 reviewed Dec 19, 2025

View reviewed changes

lucarin91 requested changes Dec 19, 2025

View reviewed changes

cmaglie requested changes Dec 19, 2025

View reviewed changes

MatteoPologruto and others added 6 commits December 19, 2025 16:32

Update service/service_stream_utility.go

b3eb0a4

Co-authored-by: Luca Rinaldi <lucarin@protonmail.com>

Update rpc/cc/arduino/flasher/v1/commands.proto

276a640

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

Update rpc/cc/arduino/flasher/v1/commands.proto

5dbb524

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

Update service/service_flash.go

f7bcd9c

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

Update service/service_flash.go

feb773b

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

Update service/service_flash.go

4ea3459

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

MatteoPologruto and others added 2 commits December 19, 2025 16:36

Update service/service_flash.go

79ef562

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

Update service/service_flash.go

5f43be3

Co-authored-by: Cristian Maglie <c.maglie@bug.st>

		// The path in which the image will be downloaded and extracted.
		string temp_path = 2;

feat: add download and flash gRPC commands #50

Are you sure you want to change the base?

feat: add download and flash gRPC commands #50

Uh oh!

Conversation

MatteoPologruto commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Change description

Additional Notes

Reviewer checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: add `download` and `flash` gRPC commands #50

feat: add `download` and `flash` gRPC commands #50

MatteoPologruto commented Dec 16, 2025 •

edited

Loading