Convert to .sublime-syntax,

[stoivo]

closing #86

As I can see the syntax is generated from MagicPython.syntax.yaml into MagicPython.tmLanguage.

I open MagicPython.tmLanguage and removed the 3 first lines

<!-- AUTOGENERATED FROM grammars/src/MagicPython.syntax.yaml -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Then I ran Plugin Development: Convert syntax to .sublime-syntax from sublime command pallet. This generated MagicPyhon.sublime-syntax.

[stoivo]

I don't know how to work with this on CI, I have little experience with Makefile. Sublime will prompt for where to save the file.

[1st1]

How can we run subl on Travis? Again, what's the point of this conversion?

[1st1]

Also, GitHub itself uses MagicPython and depends on us providing .tmLanguage file. @stoivo can you explain why we need to change MagicPython instead of updating GitSavvy?

[stoivo]

Sublime have switched all default packages to use sublime_syntax. In GitSavvy we have decided to only support the latest version of sublime and git.
I thought if would be a nice upgrade, to update to sublime-syntax. I did not know that GitHub uses tmLanguage files for language detection.

It doesn't make a problem in sublime if both tmLanguage and sublime-syntax files exist side by side. Can solve it like that.
I created a separate make task which can run locally when the sublime-syntax file needs to get updated.

[1st1]

I created a separate make task which can run locally when the sublime-syntax file needs to get updated.

Yeah, but this still doesn't work for us as it creates a build time dependency for Sublime.

In GitSavvy we have decided to only support the latest version of sublime and git.

What does GitSavvy exactly do to highlighters (or how does it interact with them)? Why does it care about what format highlighters use, tmLanguage or sublime-syntax? There are 100s of highlighters that use and will continue to use tmLanguage for years to come. Please help me to understand the real issue here.

[stoivo]

In GitSavvy we are parsing them and reading all extensions and maybe some other info which I can't remember.

We can work around this exact issue by using opening the next view with the same syntax and then when the initial file is loaded.

[stoivo]

@randy3k, do you know how we can fully automate the sublime-syntax build process? We can use some parts of https://github.com/randy3k/UnitTesting

[randy3k]

It could be done by placing a .py file and opening sublime on Travis. I could layout the exact detail if it is needed.

[stoivo]

@1st1, if I make the release process seamlessly can we merge it then?

[1st1]

@1st1, if I make the release process seamlessly can we merge it then?

Why can't you modify GitSavvy to convert old schemas to sublime-syntax on the fly? It's just that GitSavvy is the only user who requested the change, and honestly I don't want to complicate our build process just for it.

And you still didn't answer my question:

In GitSavvy we are parsing them and reading all extensions and maybe some other info which I can't remember.

Can you list that "some other info"? Because if it's just reading the extensions, then adding support of tmLanguage files to GitSavvy is a matter of half an hour work.

[stoivo]

I made a patch for GitSavvy timbrel/GitSavvy@81cc0e6

Do you want me to leave this PR with only eb6d0c8 ?

[stoivo]

??

[1st1]

I'm ooo currently. @vpetrovykh want to review eb6d0c8?

[vpetrovykh]

I am unclear what is eb6d0c8 trying to address. @stoivo, could you please explain how duplicating scopes codetags and fstring-formatting-braces from one file (MagicPython.syntax.yaml) into a couple of others is helping anything? Presumably only our own utilities actually care or know that the grammar is split over several files, but once the .tmLanguage/.cson grammar is generated every definition exists in a single file. Am I missing some subtle side-effect here? If there was a problem with the definitions missing from .cson file, why not use the latest version of MagicPython, where as far as I can tell the above scope names are present in the .cson file?

[stoivo]

If I open grammers/MagicRegExp.tmLanguage and try to run Plugin Development: convert syntax to .sublime-syntax I would get this error in the console.

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 797, in run_
    return self.run()
  File "convert_syntax in /Applications/Sublime Text.app/Contents/MacOS/Packages/Default.sublime-package", line 431, in run
  File "convert_syntax in /Applications/Sublime Text.app/Contents/MacOS/Packages/Default.sublime-package", line 358, in convert
  File "convert_syntax in /Applications/Sublime Text.app/Contents/MacOS/Packages/Default.sublime-package", line 269, in make_context
  File "convert_syntax in /Applications/Sublime Text.app/Contents/MacOS/Packages/Default.sublime-package", line 312, in make_context
Exception: no entry in repository for codetags

[stoivo]

Ping?

[vpetrovykh]

Aaaah, yes, now I see what the issue is. Thanks for noticing it. @stoivo could you please amend your pull request so that the two rules you're copying into regexp-common.inc.syntax.yaml and regexp.inc.syntax.yaml from MagicPython.syntax.yaml are instead both moved from MagicPython.syntax.yaml into regexp-common.inc.syntax.yaml? This should generate the appropriate *.cson and *.tmLanguage grammars and will avoid code duplication.

P.S. Just run make test rebuild the *.cson and *.tmLanguage grammars to make sure they indeed no longer cause you a problem when converting.
P.P.S. I'll comment on the files specifically as well.

[vpetrovykh]

The changes to the MagicRegExp grammar file have been made as part of #100. Thanks for pointing out an issue with missing rules @stoivo!

From what I understand, now that the grammar files are correct, you can use whatever conversion mechanism you like for further analysis of the grammar.

[stoivo]

Thanks, I am just back home from vacation. Awesome that you fixed it.