Atom 1.32 breaks syntax highlighting

[hermidalc]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

Python syntax highlighting is not working after update to Atom 1.32

Steps to Reproduce

1. Open any .py file in Atom 1.32

Expected behavior: Proper syntax highlighting

Actual behavior: Parts of code which should be highlighted are now broken and are white

Reproduces how often: 100%

Versions

Atom 1.32

[50Wliu]

Thanks for contributing! Can you please share some pictures or other describe in further detail the differences in syntax highlighting that you are experiencing?

[mskornilov]

It seems like the new Tree-sitter parsing system is causing this weird behaviour.
Turning it off in the settings did the trick for me.
Here's a screenshot with Tree-sitter on:

And here's with it off:

[hermidalc]

Thanks @mskornilov for posting pics of the changed syntax highlighting behavior. Mine looks the same.

[hermidalc]

Sorry now that I think about it, which syntax highlighting style (1.32 w/ tree-sitter or 1.31.2 without) is the correct one? Maybe it was broken all this time and now is fixed in 1.32?

[TheElementalOfDestruction]

• # TODO is nolonger being highlighted in pink/purple. Now it is just grey.
  

• Don't know if this one was intentional, but typing '.whatever' makes 'whatever' appear as white, and '.' appear as orange.
  

• Triple quotes ( ''' or """ ) do not start highlighting until a close triple quote appears if the first one is not the first character on the line. Instead, they turn everything white after them. They also mess up some of the highlighting before them as well. So if they appear after a space or tab, they do this.
  

• Quotes sometimes just completely crap themselves and stop suddenly after non-letter characters if no end-quote is found
  

[hermidalc]

It's definitely the Tree Sitter Parsers that causing the syntax highlighting problem, and in multiple languages. In Atom 1.32 if you go to Settings -> Core -> Tree Sitter Parsers and uncheck it highlighting behavior returns to normal. I'm actually kind of surprised something like this wasn't caught by someone before 1.32 rolled out.

[maxbrunsfeld]

The highlighting of keyword arguments has been fixed and the fix will ship in Atom 1.32.1.

[asiloisad]

the language highlithing do not deal correctly with greek letters (python 3 fully support unicode characters), so insted of [a-zA-Z] better can be [a-zA-Zα-ωΑ-Ω]

[maxbrunsfeld]

I'm going to close this out since the original issue has been fixed. @asiloisad I apologize for breaking the highlighting for identifiers with greek letters. That has been fixed (though the fix has not shipped yet).

@TheElementalOfCreation - The TODO issue is tracked at atom/language-todo#82. The issue where we were highlighting an erroneous . as a floating point number 😬 has been fixed on master. I opened a separate issue for the unmatched triple quotes: #285. Thanks for the detailed reports.

Everyone, if there are other issues with python highlighting, please open a new issue that describes the problem specifically. Thank you!

[maxbrunsfeld]

Atom 1.32.2 and 1.33.0-beta2 are out. Sorry for the delay, folks.

[hermidalc]

Hi - am I missing something or is this fix still not available? I have the latest atom with everything updated and when I turn on tree sitter parsers python highlighting still looks wrong.

Tree sitters on:

Tree sitters off:

[hermidalc]

Sorry want to ask again for some feedback... since I've been using Atom with Tree Sitter Parsers off for so long I am now unsure what would be the correct way Python should be highlighted :-/

Since I wrote my previous comment March 7, Atom or other updates have improved the Tree Sitters Python syntax highlighting to be more correct now. There still some differences so I'm guessing it was always wrong when Tree Sitters was off (and I just got used to it)?

Tree Sitters ON:

OFF:

ON:

OFF:

[50Wliu]

@hermidalc function parameters and subclasses will be highlighted again starting in (I think) Atom 1.37.0 thanks to #297 and #298 from @caleb531.

self not being highlighted is a separate issue that we haven't tackled yet.

Other variables being highlighted was removed in the tree-sitter grammar as there was no way to highlight all variables consistently (and, indeed, you'll notice the inconsistency in your own examples even with tree-sitter turned off).

[caleb531]

@hermidalc Just an update: my improvements to function/class parameter highlighting in #297 and #298 will be actually released in Atom v1.38.0—they are already in beta0, in fact—per the official release notes.

If you want them sooner, you can clone my fork of language-python and rebuild apm by running the following:

git clone https://github.com/caleb531/language-python.git
cd language-python
apm link
apm rebuild

When Atom 1.38.0 is officially released, you can later unlink and rebuild again to use the bundled language-python instead of my fork:

cd language-python
apm unlink
apm rebuild

Then restart Atom to complete the process.

[hermidalc]

@caleb531 highlighting in 1.38 looking good now. From the above two tree sitters on/off examples I had above the only possible missing hightlight now is in function def if you have e.g. *args or **kwargs the "args" and "kwargs" parts are not orange.

[50Wliu]

That was handled in #303 and should be coming down the release pipeline next :).

[Arcanemagus]

Everyone, if there are other issues with python highlighting, please open a new issue that describes the problem specifically. Thank you!

Please follow these instructions already posted if you run into further issues, thanks!