Q: back-end scenario and file contents (not just paths)

[4ntoine]

Question

I've asked on SO but did not get any answers unfortunately.
Sorry for cross posting here.

Is there any opportunity to get new files content, compiled binaries content done vis cli vis gRPC calls?

Current behavior

All the methods return arduino cli host local fs paths. This limits the use cases and require direct access
to cli host file system or alternative services to provide access (remote fs, ssh, etc).

Expected behavior

Expected to have an opportunity to pass the content of the files, not just paths. This could be done by just returning oneof depending on the (new) request arguments (paths/content).

Environment

• CLI version (output of arduino-cli version): 0.19.0
• OS and platform:

Additional context

How does it work with "arduino create"?

[silvanocerza]

The Sketch build path is generated automatically if one is not set explicitly by the user, on terminal it's done using the --build-path flag of the compile command, on gRPC using CompileRequest.build_path.

The build path is generated by this function:

arduino-cli/arduino/sketch/sketch.go

Lines 285 to 293 in 58ddae7

	func GenBuildPath(sketchPath *paths.Path) *paths.Path {
	path := ""
	if sketchPath != nil {
	path = sketchPath.String()
	}
	md5SumBytes := md5.Sum([]byte(path))
	md5Sum := strings.ToUpper(hex.EncodeToString(md5SumBytes[:]))
	return paths.TempDir().Join("arduino-sketch-" + md5Sum)
	}

The upload command, and the relative gRPC function, first searches for the binaries in the build path that would be generated using the specified Sketch path, that's only done if the --import-dir or --import-file are not specified. On gRPC they're UploadRequest.import_file and UploadRequest.import_dir respectively.

Does this help?

How does it work with "arduino create"?

Well, that's quite complicated, also it doesn't use the Arduino CLI fully as of now.

[4ntoine]

@silvanocerza thanks for the feedback. I was not clear enough. All this assumes that the client and the service are on the same host. But i'm talking about the use case when the client and the service are on different hosts. So it means not paths but actual content (in the responses) should be available over gRPC (or additional service available that returns the files content). is it possible at the moment/future? I can show what i mean in .proto files (probably even in go impl). This can make cli interesting to lots of millions lite clients (eg. mobile) users

[silvanocerza]

Interesting. 🤔

I seem to understand from the SO question that you already managed to compile a Sketch on your host so I'll assume that's working correctly.

What is not clear to me is what you're trying to accomplish after that, do you want to get the compiled binaries from the host to upload them through the client?

In any case as of now you must use the paths on the host that's running the Arduino CLI, so if you create a file on the client you must transfer it to that host to compile it and you then can upload it to a board only from the host, you can't upload from the client since it's not running the Arduino CLI.

[4ntoine]

Yup, the client can be able to upload itself (eg. OTA) or the back-end can be used for performance reason only. Can you be interested in getting a MR that supports what i propose? So it would act as is for existing use cases but will support remote back-end use cases that i'm asking about?

[silvanocerza]

Am not sure this is feasible, I think there are certain platforms that produce more than one binary. Also the size concerns me, protocol buffers are not meant to send large data sets as the official docs says.

[4ntoine]

That could be solved. For example - all build dir could be archived and returned as byte array. That could solve both multiple binary files count and size issues. In general i'd say there are multiple ways of solving it and having a separate service (eg. "filesystem" service) that would just return files bodies for the path request - that would help to separate the things and not mix everything. If it limit the paths for requests (the ones that are used for compilation and new sketches) - it would also solve possible privacy/security issue.

[4ntoine]

@silvanocerza ^ that's what i meant, would love to hear what you think about it. if it looks reasonable, i can implement and publish a MR