Improve resiliency when resolving IDEs

[fioan89]

There are a couple of cases when IDE resolving fails, sometime because Coder agent is not up and running, hence no SSH connection can be established, or some other times the Gateway Backend worker has trouble resolving different paths and environment variables. If users try to go back to the Workspaces view and select another worksapce, the IDE resolving is not even started.

A few logs to back this up:

2023-01-24 14:04:38,524 [ 86454] INFO - #c.j.g.s.GoWorkerDeployer - Standalone GTW, trying to find our plugin path
2023-01-24 14:04:38,524 [ 86454] INFO - #c.j.g.s.GoWorkerDeployer - com.jetbrains.gateway.plugin.path.for.remote.dev.workers is not set (probably launched standalone), using parent of PathManager.getLibPath=/Users/markmilligan/Library/Application Support/JetBrains/Toolbox/apps/Gateway/ch-1/223.8214.51/JetBrains Gateway.app/Contents/lib
2023-01-24 14:04:38,524 [ 86454] INFO - #c.j.g.s.GoWorkerDeployer - Resolved plugin path as /Users/markmilligan/Library/Application Support/JetBrains/Toolbox/apps/Gateway/ch-1/223.8214.51/JetBrains Gateway.app/Contents, trying to find remote-dev-workers dir
2023-01-24 14:04:38,524 [ 86454] INFO - #c.j.g.s.GoWorkerDeployer - Resolved workers dir as /Users/markmilligan/Library/Application Support/JetBrains/Toolbox/apps/Gateway/ch-1/223.8214.51/JetBrains Gateway.app/Contents/lib/remote-dev-workers
2023-01-24 14:04:38,524 [ 86454] INFO - #c.j.g.s.GoWorkerDeployer - Using remote dev worker binary from '/Users/markmilligan/Library/Application Support/JetBrains/Toolbox/apps/Gateway/ch-1/223.8214.51/JetBrains Gateway.app/Contents/lib/remote-dev-workers/remote-dev-worker-linux-amd64'
2023-01-24 14:04:38,531 [ 86461] INFO - #c.j.g.s.d.i.SshCommandExecutor - Executing on remote host: '/bin/bash -lc echo\ REMOTE_EXEC_OUTPUT_MARKER_\ &&\ echo\ $HOME'. Parameters: redirectErrorStream=false, usePty=true
2023-01-24 14:04:38,767 [ 86697] SEVERE - #c.i.o.r.u.RdCoroutineHost - Unhandled coroutine throwable
com.jetbrains.gateway.ssh.deploy.DeployException: Worker binary deploy failed: check command log for details: Should not use empty paths
at com.jetbrains.gateway.ssh.GoWorkerDeployer.deploy$intellij_gateway_core(GoWorkerDeployer.kt:238)
at com.jetbrains.gateway.ssh.GoWorkerDeployer$deploy$1.invokeSuspend(GoWorkerDeployer.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at com.intellij.openapi.rd.util.CoroutineProgressContext$Companion$create$task$1$3.invokeSuspend(BackgroundProgressCoroutineUtil.kt:172)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
at com.intellij.openapi.rd.util.CoroutineProgressContext$Companion$create$task$1.invoke(BackgroundProgressCoroutineUtil.kt:168)
at com.intellij.openapi.rd.util.CoroutineProgressContext$Companion$create$task$1.invoke(BackgroundProgressCoroutineUtil.kt:158)
at com.intellij.openapi.rd.util.CoroutineProgressContext$Companion$createBackgroundable$1$1.run(BackgroundProgressCoroutineUtil.kt:191)
at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:423)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:114)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$6(CoreProgressManager.java:474)
at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:252)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$13(CoreProgressManager.java:589)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:664)
at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:620)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:588)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:252)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalStateException: Should not use empty paths
at com.jetbrains.gateway.ssh.deploy.ShellArgument$RemotePath.(ShellArgument.kt:104)
at com.jetbrains.gateway.ssh.deploy.impl.UnixShellFacade$UnixRemotePath.(UnixShellFacade.kt:24)
at com.jetbrains.gateway.ssh.deploy.impl.UnixShellFacade$homeDir$2.invokeSuspend(UnixShellFacade.kt:61)
... 34 more