Python code completion bug

[aestes]

I am not (yet) a Zeus IDE user but am currently evaluating it. I believe that I am running the most recent release (3.97y). I have a python file I am editing and whenever I type something which would start code completion I get a debug window with several lines of python traceback ultimately ending with an error in the sphinx module which is part of the python 2.7 for windows installation.

I can't seem to paste any additional lines from the error file without tripping the spam filter in the BBS software but this outlines the problem.

I don't have a pythonpath registered in my environment but I suspect that the installed python is somehow interfering with the internal python scripting. Any suggestions on how to handle this are welcome.

Andy

Code: Select all

Scripting Error :> Traceback (most recent call last):
...
Scripting Error :>   File "C:\Python27\python-2.7.10.amd64\lib\site-packages\sphinx\__init__.py", line 47, in <module>
Scripting Error :>     def main(argv=sys.argv):
Scripting Error :> AttributeError: 'module' object has no attribute 'argv'

[aestes]

I have discovered that if I rename my c:\python27 to something like c:\python27a so that the installed python sphinx module is not getting pulled in by Zeus then the code completion works properly. This does solve the Zeus problem but breaks my other python scripts. I don't have a pythonpath environment variable so the python folder name is probably coming out of the registry.

Andy

[jussij]

Hi Andy,

As you have found out, it seems that the version of Python that ships with Zeus is running into the version of Python that you have installed.

Normally this would not cause an issues (as Zeus would just run the first Python found in the path), except that for Python code completion, Zeus uses Jedi to do the code completion, and that Jedi code is tuned to the version of Python that ships with Zeus

Any suggestions on how to handle this are welcome.

Can you tell me which version of Python you have installed in the c:\python27 folder by running this Python code:

Code: Select all

import sys
print (sys.version)

This is the version that ships with Zeus:

Code: Select all

2.7.8 (default, Sep 18 2015, 19:31:30) [MSC v.1500 32 bit (Intel)]

NOTE: The build time will be different but the version should be the same.

I don't have a pythonpath registered in my environment but I suspect that the installed python is somehow interfering with the internal python scripting.

Also can you check the PYTHONPATH as seen by Zeus by running the following Zeus script:

Code: Select all

import os
import zeus

try:
    python_path = os.environ['PYTHONPATH'].split(os.pathsep)
    zeus.message_box(0, str('\n'.join(str(path) for path in python_path)))

except KeyError:
    zeus.message_box(0, 'Error finding paths')

Just save this code to the c:\temp\test.py file and then run that file using the Zeus Macros, Execute Scripts menu.

When I run it I get this as the PYTHONPATH result:

Code: Select all

.\
C:\Program Files(x86)\Zeus\zScript\
C:\Program Files(x86)\Zeus\zScript\PyLib
C:\Program Files(x86)\Zeus\zScript\PyLib\modules
.\
C:\Program Files(x86)\Zeus\zScript\
C:\Program Files(x86)\Zeus\zScript\PyLib
C:\Program Files(x86)\Zeus\zScript\PyLib\modules
.\DLLs
.\lib
.\lib\plat-win
.\lib\lib-tk

As shown above I too have no environment PYTHONPATH defined, so you should see something similar.

Cheers Jussi

PS: While you are waiting for a fix, you can turn of the code complete by removing the triggers. To do this just use the Options, Document Types menu to edit the Python Document Type and in the Triggers section remove the triggers.

[jussij]

In addition to what I suggested in my previous post could you also try this.

Use the Zeus, Tools, DOS Command Line menu and run the following two commands:

Code: Select all

cd c:\
which python27.dll

Those commands should report this result:

Code: Select all

cd c:\
C:\>which python27.dll
C:\Program Files (x86)\Zeus\python27.dll

Cheers Jussi

[aestes]

Jussi,

Python version is

Code: Select all

2.7.8 (default, Jun 30 2015, 18:15:33) [MSC v.1500 32 bit (Intel)]

This is the result of the script (transcribed, I can't seem to copy from the dialog box):

Code: Select all

.\
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\modules
C:\Program Files (x86)\Zeus\zScript\
C:\Program Files (x86)\Zeus\zScript\PyLib
C:\Program Files (x86)\Zeus\zScript\PyLib\modules
.\
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib
C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\modules
C:\Program Files (x86)\Zeus\zScript\
C:\Program Files (x86)\Zeus\zScript\PyLib
C:\Program Files (x86)\Zeus\zScript\PyLib\modules
.\DLLs
.\lib
.\lib\plat-win
.\lib\lib-tk

Finally, the which command gave me C:\Program Files (x86)\Zeus\python27.dll

Andy

[jussij]

Thanks for that feedback.

This is all a bit strange, since your feedback suggest this file should never be found:

Code: Select all

C:\Python27\python-2.7.10.amd64\lib\site-packages\sphinx\__init__.py

But of course it was

I'll take a closer look at the Python code to see does some sort of special search for files at the C:\Python27\ location.

But I'm total at a loss as to why and how it actually managed to find that file

I don't understand what made Python to go looking for a file in this location

Code: Select all

C:\Python27\python-2.7.10.amd64\

Thanks again for the feedback.

Cheers Jussi

[aestes]

Jussi,

It is a bit curious. I do not have c:\python27 in my path so it isn't finding it that way and it isn't in my pythonpath so it must be going through the registry. I actually have several versions of python on my system (anaconda 3.4, python 2.7, and ironpython 2.7 as well as a couple of embeded python scripting engines) but I haven't run into a case like this. I suspect that since Zeus is using (more or less) the same version that I have installed in c:\python27 then it must be going through the registry. I do have an entry at HKCU\Software\Python\PythonCore\2.7 which points into c:\python27.

Do you have the sphinx package loaded in your python environment? I found that if I keep c:\python27 as is and rename ...\site-packages\sphinx to sphinxa then the code completion works without error.

Andy

[jussij]

Hi Andy,

Thanks for posting your feedback. Much appreciated.

I do have an entry at HKCU\Software\Python\PythonCore\2.7 which points into c:\python27.

I suspect something in the Zeus's version of Python might be finding that registry key.

Do you have the sphinx package loaded in your python environment?

I don't think so.

Zeus only has a fairly minimal Python install, the components of which can be found here:

Code: Select all

C:\Users\<Current User Name>\AppData\Roaming\Xidicone\Zeus\zScript\PyLib

Code: Select all

I found that if I keep c:\python27 as is and rename ...\site-packages\sphinx to sphinxa then the code completion works without error.

I vaguely remember Zeus having issues with sites many versions ago.

I'm not really up to speed on how the site stuff works in Python but I think I had to add this file to the Zeus install to get things to work:

Code: Select all

C:\Users\<Current User Name>\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\site.py

Maybe something in that that file is causing this issue

I'll so some research into this 'site' stuff to see if Zeus is doing something wrong.

Cheers Jussi

[aestes]

I think I see what is happening and it isn't anything Zeus is doing. It looks like the following:
You call into jedi_auto_complete, ultimately jedi (from docstrings.py) tries to load numpy to get NumpyDocString, NumpyDocString then tries to load sphinx. If Jedi can't find numpy it catches a loaderror exception (line 44 of jedi\evaluate\docstrings.py) and makes other arrangements. The problem occurs when numpy is available; it then tries to load sphinx. Sphinx tries to reference sys.argv which I suspect doesn't exist in your scripting engine so it throws an exception (AttributeError). I can fix the issue locally on my machine by adding an except AttributeError: in jedi\evaluate\docstrings.py at line 47 (and copying the two lines from the except ImportError: handler just above it). Of course, this could be on the next upgrade of Zeus (not sure how your upgrades work). It does slow down the load of the autocomplete slightly but allows the rest of the python implementation to work properly. A different fix might be to set sys.argv based on the Zeus command line but I don't know what sphinx will do with it.

In any event, I think I have a work-around on my side but this will pop up for anyone else who uses Zeus for Python and has a full python installation (which also happens to be found from the pythonpath environment or registry).

Andy

[jussij]

Hi Andy,

Once again, thanks for your thorough analysis

Sphinx tries to reference sys.argv which I suspect doesn't exist in your scripting engine

Using the information from your analysis I found this related link: https://mail.python.org/pipermail/pytho ... 21095.html

It looks like it is a bit of a known issue for embedded systems

A different fix might be to set sys.argv based on the Zeus command line but I don't know what sphinx will do with it.

Can you tell me if this .\Zeus\zScript\jedi_auto_complete.py change also fix the issue

Code: Select all

#... Line 16 change
import sys
import zeus

#... Line 42 change
    sys.argc = 0
    sys.argv = None

If that works then I can make a similar change inside of the Zeus Python module itself by using code something like this:

Code: Select all

Py_Initialize();
int argc = 0;
char **argv = 0;
PySys_SetArgv(argc, argv);

I can fix the issue locally on my machine by adding an except AttributeError: in jedi\evaluate\docstrings.py at line 47 (and copying the two lines from the except ImportError: handler just above it).

Is this the docstrings.py change you are planning to make

Code: Select all

  
try:
    from numpydoc.docscrape import NumpyDocString
except ImportError:
    def _search_param_in_numpydocstr(docstr, param_str):
        return []
# Zeus Start of Change - Ignores the AttributeError
except AttributeError:
    def _search_param_in_numpydocstr(docstr, param_str):
        return []
# Zeus End Of Change
else:

Cheers Jussi

[aestes]

Unfortunately it doesn't look like it is that easy The series of errors are started by line 18 of jedi_auto_complete.py when it imports jedi_code so the fix happens too late.

Code: Select all

---------------------------------------------------------------------------
      Zeus - Version 3.97y
      Copyright (c) Xidicone P/L 1993-2015.  All rights reserved.
---------------------------------------------------------------------------
 **** Unregistered Software. To be used for evaluation purposes only. ****
---------------------------------------------------------------------------


Scripting Error :> Traceback (most recent call last):
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\jedi_auto_complete.py", line 18, in <module>
Scripting Error :>     import jedi_code
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\jedi_code.py", line 19, in <module>
Scripting Error :>     import jedi
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\__init__.py", line 41, in <module>
Scripting Error :>     from jedi.api import Script, Interpreter, NotFoundError, set_debug_function
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\api\__init__.py", line 24, in <module>
Scripting Error :>     from jedi.api import keywords
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\api\keywords.py", line 6, in <module>
Scripting Error :>     from jedi.evaluate import compiled
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\evaluate\__init__.py", line 68, in <module>
Scripting Error :>     from jedi.evaluate import representation as er
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\evaluate\representation.py", line 44, in <module>
Scripting Error :>     from jedi.evaluate import docstrings
Scripting Error :>   File "C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\jedi\evaluate\docstrings.py", line 43, in <module>
Scripting Error :>     from numpydoc.docscrape import NumpyDocString
Scripting Error :>   File "C:\Python27\python-2.7.10.amd64\lib\site-packages\numpydoc\__init__.py", line 3, in <module>
Scripting Error :>     from .numpydoc import setup
Scripting Error :>   File "C:\Python27\python-2.7.10.amd64\lib\site-packages\numpydoc\numpydoc.py", line 21, in <module>
Scripting Error :>     import sphinx
Scripting Error :>   File "C:\Python27\python-2.7.10.amd64\lib\site-packages\sphinx\__init__.py", line 47, in <module>
Scripting Error :>     def main(argv=sys.argv):
Scripting Error :> AttributeError: 'module' object has no attribute 'argv'

PYTHONPATH=.\;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\modules;C:\Program Files (x86)\Zeus\zScript\;C:\Program Files (x86)\Zeus\zScript\PyLib;C:\Program Files (x86)\Zeus\zScript\PyLib\modules;.\;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib;C:\Users\adestes\AppData\Roaming\Xidicone\Zeus\zScript\PyLib\modules;C:\Program Files (x86)\Zeus\zScript\;C:\Program Files (x86)\Zeus\zScript\PyLib;C:\Program Files (x86)\Zeus\zScript\PyLib\modules;.\DLLs;.\lib;.\lib\plat-win;.\lib\lib-tk

Macro script generated '78' Debug, Error and/or Warning message(s).

Andy

[jussij]

Hi Andy,

Could you try the new scriptpy.dll found here: http://www.zeusedit.com/z300/scriptpy.zip

Just rename the older one out of the way as a backup and then unzip the new replacement dll:

Code: Select all

c:\Program Files (x86)\Zeus\scriptpy.dll => c:\Program Files (x86)\Zeus\scriptpy.old.dll

This new version sets argc to one and the argv to an empty string on initialization.

The new dll will report as version2.04 in the Options, Editor Options menu, Scripting panel.

With a bit of luck it might fix the issue

Cheers Jussi

[aestes]

Jussi,

Yes, that appears to have fixed the issue! I will play around with it today and see if I see that causes any other issues (although I don't see why it should).

Thank you for the all the work!

Andy