Special Characters Change When Saving File

[AlanStewart]

Hello Jussi,

So, I've had this problem for a long time and I'm kinda confused as to why it's happening. The issue has to do with special characters in files not displaying properly when the file is saved and reloaded. Everything works fine when I'm editing over FTP into a Linux machine, but when I copy the same file onto my Windows box, the file gets corrupted.

For example, this piece of text "Rémi" [attachment=1]remi_1.png[/attachment]
is just fine working in Linux but in Windows, when I first load the file up, everything is just fine, but when I make a change and save the file, a Reload shows that all the special characters are now little filled boxes "R‚mi" [attachment=0]remi_2.png[/attachment].

File type is ANSI and EOL marker is Unix (just the \n I believe). When I try to change the file type to UNICODE and UTF-8 things get even weirder so I don't think that's where the problem is.

Any idea what the is issue is and what I might be able to do to fix it?

Thanks,
Alan Stewart

P.S. This is 3.98fb5. I don't have a code for 3.98o and the link to 3.98j just points to 3.98o.

[jussij]

Hi Alan,

but when I copy the same file onto my Windows box, the file gets corrupted.

How are you copying the files to the machine

File type is ANSI and EOL marker is Unix (just the \n I believe).

I'm not sure this is correct

Since you are using a Linux machine the files on that machine will most likely be in UTF8 format.

On Windows text files tend to default to ASCII not UTF8 and that difference can cause differences in display.

So the first thing I would do is keep an eye on the status bar to see what type of text file you are editing.

I would also look to create a Zeus document type that has the UTF8 file type enabled.

Take a look at the Go document type. That document type always creates a UTF8 document.

Cheers Jussi

[AlanStewart]

How are you copying the files to the machine

WinSCP to copy from Linux to Windows. Then just ^C ^V to move the Linux file into a different directory.

I'm not sure this is correct

That's the Save As options. I was hoping that I could find an option to save them as that would work. There is no Save As ASCII option.

So the first thing I would do is keep an eye on the status bar to see what type of text file you are editing.

Oddly enough, ALL my currently loaded files are all ASCII and CrLf or just Lf.

I would also look to create a Zeus document type that has the UTF8 file type enabled.

That I haven't done. I've always just created natively or by copying an existing file. Lemme see what I can do.

Take a look at the Go document type. That document type always creates a UTF8 document.

Huh. Mine are all ANSI. That's bizarre. Am I creating them incorrectly?

Okay, I created a file. Default was ASCII but I did a Save-As as UTF-8, then copied the file in question over and had to re-save it. This time it went from "Rémi" to "RAcmi" and the type switched back to ASCII. That's odd.

[jussij]

Hi Alan,

This is what is going wrong:

1. On Linux text files default to UTF8 without a BOM marker
2. On Windows text files default to ASCII and UTF-8 files need a BOM marker
3. WinSCP will copy the files down in binary mode so the actual contents of the Linux and the Windows files will be identical.
4. However since Windows (and Zeus) default to opening file as ASCII the file will not always displays correctly.

There is no Save As ASCII option.

There is nothing to save. The files on the two machines will be identical.

But you have to tell Zeus the file is UTF-8 because by default Windows text files as ASCII.

Now I just tried setting the UTF-8 option in the default document type but that setting does not seem to flow through to the other document types as it should.

This looks like a bug

However, I know this does work for specific document types because that is how the Go document type is configured.

As such you will need edit the document type these files and set them as UTF-8 with No BOM.

As a simple test take one of your broken files and do a Save As to a file with a .Go file extension.

By doing this the file will be now be recognised as UTF-8 with No BOM and should display correctly.

Cheers Jussi

[AlanStewart]

Now I just tried setting the UTF-8 option in the default document type but that setting does not seem to flow through to the other document types as it should.
This looks like a bug

Well that's a bummer

As such you will need edit the document type these files and set them as UTF-8 with No BOM.

Is this something I'll want global to all my .CSS files or should I just try and set it for one specific file? Will Zeus be able to figure out what the file is without a BOM?

As a simple test take one of your broken files and do a Save As to a file with a .Go file extension.
By doing this the file will be now be recognised as UTF-8 with No BOM and should display correctly.

I tried that. I was still ASCII. With a bit more playing around I got it to work as UTF-8 but it still came back as "RAcmi". Come to think of it, I don't think I've ever had this work properly on a Windows box. I just kinda ignored it for the most part.

[jussij]

Is this something I'll want global to all my .CSS files or should I just try and set it for one specific file?

You have to set it globally (i.e. mark the Zeus CSS document type as UTF-8 with No BOM).

You can't set it on a per file basis because doing that mean would mean adding a BOM to the file.

While that BOM would work fine on Windows (and in Zeus) on your Linux machine most of the Linux programs would reject a file that contains a BOM and that is because Linux assume all files are UTF-8 so no BOM is ever required.

So putting BOM markers into files will start to break things in the Linux environment.

Will Zeus be able to figure out what the file is without a BOM?

Only if you tell Zeus the document is UTF-8 via that document type setting.

That Zeus document type setting is effectively is the same as a BOM

Cheers Jussi