CodeMirror - Code Editor within your browser

A topic built around all the best posts and information from the site. It includes informative posts, links to resources, tips, tricks, ect... Do not post here unless it will become a resource based post.
Post Reply
User avatar
a_bertrand
Posts: 1537
Joined: Mon Feb 25, 2013 1:46 pm

CodeMirror - Code Editor within your browser

Post by a_bertrand »

CodeMirror is a Javascript library used to create a code editor directly inside your browser. Now I don't suggest you to let your game admins edit directly your PHP files! That could be quite dangerous and let open the door of all sorts of hacks and nasty things, however the editor could be used for quite a few things and edit many of the programming languages without having to edit yourself the syntax files.

https://codemirror.net/

Inside my engine for example I use it to let the game owner edit the scripts which are nearly Javascript. Nearly because I wanted to keep a similar syntax, avoiding that people need to learn yet another language, easier to find info and help and also to re-use the existing syntax highlight. It is not the full Javascript and for sure I don't just "eval" the text the game owner writes. If you are interested how I made that language drop a line and I shall make a post about it too.

Anyhow to come back to code mirror, I integrated it inside my engine and you can check a demo here:

https://www.dotworldmaker.com/Home/code_editor.html

What I did is integrate a real-time syntax checker, and you can try it by writing garbage in the code and a yellow line on top should appear saying the error as well as highlight the line where the checker think the error is. Also if you move the cursor (either by clicking or with the arrow keys) over an API function a little window with a small help will appear. Finally the last piece was the code completion which works by checking what you have under the cursor and trying to suggest API functions. Try to write player. with the dot included and then you can scroll up / down with the arrow keys and select the function you want to add (by pressing return or tab).

How it works:
The syntax checker is nothing else than my parser run every time you change the code and the error output is then grabbed and it will highlight the line number which is wrong using CodeMirror api "editor.addLineClass".

The inline help works by hocking myself to the "cursorActivity" event of CodeMirror extracting the word at the cursor position and if the API call is found show a small DIV with the content of the help.

The code completion is actually part of the "cursorActivity" event and if the API call is not found, try to suggest something. However I had then to use the "extraKeys" options and define "Enter", "Tab", "Up", "Down" and "Esc" keys.

Conclusion:
CodeMirror is certainly a really useful lib, however I had more than some issues to find the right way to do what I wanted maybe due to the manual which is somewhat odd to read at first or maybe because I thought I could solve it differently. Anyhow after all that I do finally have the tool I wanted: a good code editor within my engine.
Creator of Dot World Maker
Mad programmer and annoying composer
User avatar
Jackolantern
Posts: 10893
Joined: Wed Jul 01, 2009 11:00 pm

Re: CodeMirror - Code Editor within your browser

Post by Jackolantern »

Very cool! I have looked at a few JS-based editors for integrating into a web product but this definitely looks like the most full-featured I have seen. Even with some issues to work out, the amount of work saved still has to be massive.
The indelible lord of tl;dr
User avatar
hallsofvallhalla
Site Admin
Posts: 12031
Joined: Wed Apr 22, 2009 11:29 pm

Re: CodeMirror - Code Editor within your browser

Post by hallsofvallhalla »

Very sweet!
User avatar
a_bertrand
Posts: 1537
Joined: Mon Feb 25, 2013 1:46 pm

Re: CodeMirror - Code Editor within your browser

Post by a_bertrand »

Creating an editor of that level would have taken me loads of time, which of course would have been removed from other parts. So it's definitively a great saver and the end result even if it's not Visual Studio, is still pretty cool in my opinion.

Ideally I would want to show all the errors for example and not just the first encountered, but that's an issue with my parser which stops at the first error not the code editor.

I could also implement a toolbar for the editor, and maybe an auto-formating... as always there is nearly always place for improvements ;)
Creator of Dot World Maker
Mad programmer and annoying composer
User avatar
Jackolantern
Posts: 10893
Joined: Wed Jul 01, 2009 11:00 pm

Re: CodeMirror - Code Editor within your browser

Post by Jackolantern »

Now compilers I know a little bit about (not exaggerating...just a little bit lol). And yes, showing multiple errors is difficult, particularly with something like a recursive descent compiler. I'm not sure what kind of compiler you have written. I had to write a C compiler in school that reported multiple errors. I did it with a recursive descent compiler. I added an additional layer to the lexer (I think if I remember right it is called a "lexer-parser layer" since it has some attributes of both sides of the compiler) that groups what should be complete statements into a higher-order token. These tokens stand on their own, not in the hierarchy of the standard tokens. So say you have this code:

Code: Select all

for (x = a + 4; x < (getTheCount() + 3); x++) {
    //other statements
}
The lexer-parser layer's goal is to break out everything that could be a complete statement and create an alternative list (not a hierarchy) of these higher-order tokens. They would, in this case be this:

Code: Select all

for() {
} 

Code: Select all

x = a + 4

Code: Select all

x < (substitute) 

Code: Select all

getTheCount() + 3

Code: Select all

x++ 
Honestly I think I broke that up rather sloppily (like getTheCount() alone is a statement which would have yielded "substitute + 3", etc. but I am sure you get the point). You then add a reference to these higher-order tokens to the first token (or last, depending on your compiler design) in the actual token sting they originated from. So the first higher-order token for the FOR structure would have a reference to the "for" token (or the final curly brace).

How these are used is that, if you find an error in a statement and you are inside of at least one open statement (you are in a nested statement), you can start to work your way out, using the higher-order tokens to find variables out of scope, syntax errors and other things until you reach a point where you are at 0 open statements, at which point you know the next code has to be the opening to another statement and you can get back on track. The point of it is to try to isolate the error as much as possible to keep the rest of the code around it as parsable as possible.

This method is definitely not perfect and this was the second method we learned to try to report multiple errors (the first being to just ignore everything following the error until all statements were closed and start over). First, it doesn't report them all. Taking each statement in isolation like that means that some of the semantic meaning is lost so not all types of errors can be caught the same way you can catch your first error. Also, just a complete train wreck of grammar is going to throw-off the lexer-parser layer and you basically have to write up a whole new mechanism to handle lexer-parser layer errors and it is quite difficult to report all errors at that level.

I honestly don't think many real compilers use this style of error recovery and detection, but then again most real compilers out there aren't going to use anything taught in an intro to compilers class lol.
The indelible lord of tl;dr
User avatar
a_bertrand
Posts: 1537
Joined: Mon Feb 25, 2013 1:46 pm

Re: CodeMirror - Code Editor within your browser

Post by a_bertrand »

Many many thanks for the info Jackolantern, I must say I'm not all that of an expert in this field, and wonder actually how much sense it makes to invest time to try to solve those issues. As you point out showing the errors after the first one tends to be less interesting / correct specially if the syntax break after the first one and your other issues are just pseudo errors generated by the first one.

I think for the moment I will skip and concentrate myself on other tasks (so many anyhow ;) ), but thanks for pointing me to interesting ideas / concepts.
Creator of Dot World Maker
Mad programmer and annoying composer
User avatar
Jackolantern
Posts: 10893
Joined: Wed Jul 01, 2009 11:00 pm

Re: CodeMirror - Code Editor within your browser

Post by Jackolantern »

No problem! It is the least I can do for all of the gems you have provided to the community :)

But yeah, I would probably recommend to not worry about showing multiple errors for right now, particularly if it will require a major restructuring of your compiler (which it usually does). If I recall correctly, I think there are still quite a few widely-used compilers out there today that don't report multiple errors at a time and they get by just fine. I think most devs out there accept that fixing one error may show another because it happens even on highly-advanced, multiple-error-reporting compilers, such as the .NET compiler and Java compiler. It may happen for different reasons in those cases, but it still happens.

I think, like you said, your time could be better spent elsewhere and then maybe come back to it down the road if your users ever specifically bring it up as an issue or if that mythical day ever arrives that all developers hope for and you have some spare time (you know, that day that we are going to do all that refactoring, updating core dependencies, etc.) 8-)
The indelible lord of tl;dr
Post Reply

Return to “Resources”