Special Characters Change When Saving File

If reporting a bug with the Zeus IDE please post the details here. Please do not post questions here.
Post Reply
AlanStewart
Posts: 82
Joined: Fri Jun 02, 2006 6:52 pm

Special Characters Change When Saving File

Post by AlanStewart » Tue Jun 12, 2018 11:31 pm

Hello Jussi,

So, I've had this problem for a long time and I'm kinda confused as to why it's happening. The issue has to do with special characters in files not displaying properly when the file is saved and reloaded. Everything works fine when I'm editing over FTP into a Linux machine, but when I copy the same file onto my Windows box, the file gets corrupted.

For example, this piece of text "Rémi" [attachment=1]remi_1.png[/attachment]
is just fine working in Linux but in Windows, when I first load the file up, everything is just fine, but when I make a change and save the file, a Reload shows that all the special characters are now little filled boxes "R‚mi" [attachment=0]remi_2.png[/attachment].

File type is ANSI and EOL marker is Unix (just the \n I believe). When I try to change the file type to UNICODE and UTF-8 things get even weirder so I don't think that's where the problem is.

Any idea what the is issue is and what I might be able to do to fix it?

Thanks,
Alan Stewart

P.S. This is 3.98fb5. I don't have a code for 3.98o and the link to 3.98j just points to 3.98o.
Attachments
remi_2.png
remi_2.png (191 Bytes) Viewed 2223 times
remi_1.png
remi_1.png (199 Bytes) Viewed 2223 times

jussij
Site Admin
Posts: 2516
Joined: Fri Aug 13, 2004 5:10 pm

Re: Special Characters Change When Saving File

Post by jussij » Wed Jun 13, 2018 1:11 am

Hi Alan,
but when I copy the same file onto my Windows box, the file gets corrupted.
How are you copying the files to the machine :?:
File type is ANSI and EOL marker is Unix (just the \n I believe).
I'm not sure this is correct :?:

Since you are using a Linux machine the files on that machine will most likely be in UTF8 format.

On Windows text files tend to default to ASCII not UTF8 and that difference can cause differences in display.

So the first thing I would do is keep an eye on the status bar to see what type of text file you are editing.

I would also look to create a Zeus document type that has the UTF8 file type enabled.

Take a look at the Go document type. That document type always creates a UTF8 document.

Cheers Jussi

AlanStewart
Posts: 82
Joined: Fri Jun 02, 2006 6:52 pm

Re: Special Characters Change When Saving File

Post by AlanStewart » Wed Jun 13, 2018 11:13 pm

jussij wrote:How are you copying the files to the machine :?:
WinSCP to copy from Linux to Windows. Then just ^C ^V to move the Linux file into a different directory.
jussij wrote:I'm not sure this is correct :?:
That's the Save As options. I was hoping that I could find an option to save them as that would work. There is no Save As ASCII option.
jussij wrote:So the first thing I would do is keep an eye on the status bar to see what type of text file you are editing.
Oddly enough, ALL my currently loaded files are all ASCII and CrLf or just Lf.
jussij wrote:I would also look to create a Zeus document type that has the UTF8 file type enabled.
That I haven't done. I've always just created natively or by copying an existing file. Lemme see what I can do.
jussij wrote:Take a look at the Go document type. That document type always creates a UTF8 document.
Huh. Mine are all ANSI. That's bizarre. Am I creating them incorrectly?

Okay, I created a file. Default was ASCII but I did a Save-As as UTF-8, then copied the file in question over and had to re-save it. This time it went from "Rémi" to "RAcmi" and the type switched back to ASCII. That's odd.

jussij
Site Admin
Posts: 2516
Joined: Fri Aug 13, 2004 5:10 pm

Re: Special Characters Change When Saving File

Post by jussij » Thu Jun 14, 2018 12:22 am

Hi Alan,

This is what is going wrong:
  1. On Linux text files default to UTF8 without a BOM marker
  2. On Windows text files default to ASCII and UTF-8 files need a BOM marker
  3. WinSCP will copy the files down in binary mode so the actual contents of the Linux and the Windows files will be identical.
  4. However since Windows (and Zeus) default to opening file as ASCII the file will not always displays correctly.
There is no Save As ASCII option.
There is nothing to save. The files on the two machines will be identical.

But you have to tell Zeus the file is UTF-8 because by default Windows text files as ASCII.

Now I just tried setting the UTF-8 option in the default document type but that setting does not seem to flow through to the other document types as it should.

This looks like a bug :(

However, I know this does work for specific document types because that is how the Go document type is configured.

As such you will need edit the document type these files and set them as UTF-8 with No BOM.

As a simple test take one of your broken files and do a Save As to a file with a .Go file extension.

By doing this the file will be now be recognised as UTF-8 with No BOM and should display correctly.

Cheers Jussi

AlanStewart
Posts: 82
Joined: Fri Jun 02, 2006 6:52 pm

Re: Special Characters Change When Saving File

Post by AlanStewart » Thu Jun 14, 2018 1:30 am

jussij wrote:Now I just tried setting the UTF-8 option in the default document type but that setting does not seem to flow through to the other document types as it should.
This looks like a bug :(
Well that's a bummer
jussij wrote:As such you will need edit the document type these files and set them as UTF-8 with No BOM.
Is this something I'll want global to all my .CSS files or should I just try and set it for one specific file? Will Zeus be able to figure out what the file is without a BOM?
jussij wrote:As a simple test take one of your broken files and do a Save As to a file with a .Go file extension.
By doing this the file will be now be recognised as UTF-8 with No BOM and should display correctly.
I tried that. I was still ASCII. With a bit more playing around I got it to work as UTF-8 but it still came back as "RAcmi". Come to think of it, I don't think I've ever had this work properly on a Windows box. I just kinda ignored it for the most part.

jussij
Site Admin
Posts: 2516
Joined: Fri Aug 13, 2004 5:10 pm

Re: Special Characters Change When Saving File

Post by jussij » Thu Jun 14, 2018 3:44 am

Is this something I'll want global to all my .CSS files or should I just try and set it for one specific file?
You have to set it globally (i.e. mark the Zeus CSS document type as UTF-8 with No BOM).

You can't set it on a per file basis because doing that mean would mean adding a BOM to the file.

While that BOM would work fine on Windows (and in Zeus) on your Linux machine most of the Linux programs would reject a file that contains a BOM and that is because Linux assume all files are UTF-8 so no BOM is ever required.

So putting BOM markers into files will start to break things in the Linux environment.
Will Zeus be able to figure out what the file is without a BOM?
Only if you tell Zeus the document is UTF-8 via that document type setting.

That Zeus document type setting is effectively is the same as a BOM ;)

Cheers Jussi

Post Reply
kuchnie warszawa opinie

Who is online

Users browsing this forum: No registered users and 6 guests