[Discussion] Milestone 2.2.0

[vors]

First of all, I want to share some highlights of the recent release 2.1.0

• New unit test framework allows describe fundamental rules about syntax scopes. Now ST scopes can be compared to tokens, produced by PS parser (used for powershell_ise and PSReadLine highlighting). Credits to @guillermooo for introducing unit tests.
• Tons of bugfixes for syntax. Credits to @bcrotty for his contribution. Credits to @Jaykul for his contribution for previous release 2.0.0 and support for PS v3.
• Github syntax highlight library will use our work for all github powershell code! I don't have information when it will happen, but eventually we will impact experience for all open source powershell developers. I'm quite excited about it.

For next release, there are few areas where we can improve.

• Better test automation to reduce risk of regressions. Currently, unit tests have quite few ignored errors.
• New language features, like Add classes syntax support (powershell v5) #42 and Add .mof files syntax support (powershell v4) #43
• Clean up old bugs (older then 2015): fix/close.
• Make goto symbol feature usable symbols list for powershell syntax #27

I encourage you to share opinions, what you want to see in the next release.

Then, I will create a milestone 2.2.0 with general description of milestone's goals and assign actionable items to it.

[guillermooo]

Excellent @vors! Thanks for taking the lead here :)

Generally speaking, I think it'd be worth adding the ability of running PS code from ST. For example, PowershellUtils lets you write a short script and have the output populate each ST text region in the buffer.

Also, perhaps this package could expose some sort of command/api so that other packages can execute PS code. For instance, I'd like Vintageous (Vim emulation) to use PowerShell as a shell language for some Vim commands.

Anyway, nothing too specific, just throwing some ideas around.

[vors]

@guillermooo excellent ideas!

[vors]

I also was thinking about a color schema that supports all PowerShell scopes. It can be in colors of ISE :)

[bcrotty]

I've had the same idea lately! I really like the color theory and thought that went into creating Solarized (although the theme itself is not my first choice). http://ethanschoonover.com/solarized

[vors]

btw, github already use our grammar for all PS code!

[guillermooo]

👏 😎

[vors]

More exciting news!

@jugglingnutcase is the author of PowerShell laguage package for atom text editor. (btw, 4k downloads). Atom and SublimeText share the same grammar, and he is considering using our repo as a vendor submodule for grammar.

His package also has a pretty big unit test coverage, which we can leverage.

I want to talk about tests a little bit more. When I prepared pull request to lingust, I spent some time fixing grammar to be compatible with https://lightshow.githubapp.com/. You can see changes in PR #75. Since now we have github as a customer, we should make sure that linguist library works well with all new releases. Same for atom. We may want make grammar unit testing environment-agnostic: run it outside of SublimeText. Maybe lingust lib has some infrastructure for that.

We can think about sharing other features (goto symbol, snippets) between two editors.

[jrsconfitto]

i'll throw my thoughts out there.

⚠️ i often don't know what i'm talking about and i'm doing this quick, so i may not be very clear atm ⚠️

i'm 👍 for making the grammar unit testing environment-agnostic, but i'm not sure which platform is best for that. i don't know how SublimeText deals with the grammar files... is there a library they're using for that that is open source that we could use for automated testing? i know very little python so if it's python, i'll need lots of help.

Since i'm biased 😉, i suggest that we utilize node and npm for our tests. There's a bunch of benefits to that:

1. good cross-platform support via node/npm
2. js is fairly approachable and can make it easier for current and future contributers
3. converting between the plist and js is pretty simple and i already have code doing this
4. we can use tests i've already written (after adjusting them for this new grammar)
5. apm and Atom's grammar library (first-mate) are both on npm (apm on npm & first-mate on npm) so that makes them easy to pull down in isolation for automated testing.

i'm not sure how to test how the grammar will behave in other environments, though it looks like GitHub.com will act similarly to Atom because they both are based on textmate for grammars.

[vors]

@jugglingnutcase As far as I know sublime exposes all APIs via sublime.py module which calls sublime_api.py (native code). To get grammar scopes you need to 1) load sublime.py (I don't know how to do it outside of sublime) 2) Call set_syntax_file() 3) call some api to get scopes like find_by_selector.

Let me tell you a little bit more about our current tests. There are currently two types: in-proc tests, there are very few of them. And out-of-proc pester tests. By proc I mean sublime text process. Out-of-proc tests consume text files, produced by specific in-proc tests, that describes scopes. The reason to make them out-of-proc is to use tokens, generated by PowerShell parser as canonical scopes. So we can make sure that not only particular inputs produce pre-determined scopes, but we can verify fundamental principals for scopes against language parser. Currently there are 3 such principles (in release 2.1.0).

1. Scopes should be the same for input and for input.ToUpper()
2. Scopes should be the same for input and for input.ToLower()
3. Scopes should not break powershell parser tokens apart. Sample from a bug, fixed in 2.1.0 Get-Not should not be highlighted as Get** and -Not**.
   First two doesn't really require PowerShell parser tokens, but I tested them out-of-proc for consistency.

[vors]

There are two directions that I'm personally interested in:

1. Continue work on grammar
   a. Leverage atom tests
   b. Test grammar completely out-of-proc.
   c. Fix bugs, provide go-to definition (can it be shared with atom?).

2. Create a proof-of-concept prototype: SublimeText-powershell as an alternative for powershell_ise
   a. I want to be able to open special kind of tab: powershell REPL, with expirience, similar to powershell_ise.
   b. We probably need to implement custom powershell host for that or implement scraping of existing host (can be harder).
   c. Ideally think about project similar to http://www.omnisharp.net/

What do you think about it?

[jrsconfitto]

Interesting. i'll have to take a closer look at the tests in this project to make sure i understand what's going on.

i like the directions you're considering, those sound like really useful things.

a. Leverage atom tests

How do you see this working out? Translating them to the pester tests you have?

[vors]

I don't know yet. But I don't want sacrifice test coverage without good reason. Good reason can be that old tests are completely irrelevant with new grammar.

PowerShell/Pester was picked semi-randomly. It's a convenient and cheap way to call parser API. But it could be C# or F# as well. PowerShell is not my top-choice for writing big things.

But one advantage or writing tests in PowerShell is dogfooding. Also it's very likely that anybody who want to run them will have everything on the box already. Except maybe pester, but it's a one command to install from PowerShellGet, PSGet or chocolaty.

[guillermooo]

I also was thinking about a color schema that supports all PowerShell scopes. It can be in colors of ISE :)

Shall we create a new repo for the color scheme?