tourmaline wrote:Any async operations are part of node then. Just passing a function as an argument does not make a method async, yes/no?
No, just passing a function as an argument does not make it async. If you did this:
Code: Select all
function notAsync(a, b, callback) {
callback(a + b);
}
notAsync(1, 2, function(result) {
console.log(result);
});
There is nothing async about this. What I mean by "async" to be specific is when execution will go ahead and do something else while it is waiting for the true async function call to return. But that doesn't happen here. This code would executed procedurally, and nothing else would be done from the time it starts to the time it ends.
tourmaline wrote:And if that is true, what about something simple like fs.readFile and fs.readFileSync. What would be the detriment to using the sync version?
Sync (or "blocking") versions of I/O operations being included in node is controversial, because you should
never call them in any kind of server that is handling concurrent users. Doing so would kill the performance of your server. That is because the sync versions of the I/O functions halt the program until the result is returned, just like what happens in Java, C++, and most other languages when you make normal, blocking I/O calls. That is why those traditional languages resort to multithreading for concurrency. Multithreading is far more complex and requires much more planning than asynchronous programming.
The sync versions are mostly included for people to create small, single-purpose tools and utility scripts in node. But what is controversial is that in no case are they ever required. Those small utility scripts can still use the async versions, but I guess when you don't need concurrency, blocking I/O is more simple.
tourmaline wrote:I don't see where the process running ahead, and then trying to give a result back to the process at a later point makes any sense. I'm apparently not seeing all the benefits to asynchronous programing.
The benefits of asynchronous programming become much more apparent when they are matched with an "event loop", which node is. For example, in a real-time website application, you have events set up to serve various webpages, and you may also have Socket.io events set up to handle socket connections, socket messages, etc. When a request for a webpage comes in, the code inside the event will make an async call to the file system to retrieve the webpage code. Now since it is async, it doesn't wait for the slow hard drive to retrieve the webpage. It is free to go and begin servicing other website requests, Socket.io events, etc. And when the system is notified that the website code has arrived from the hard drive, possibly 3 or 4 events were processed in the meantime, but node jumps back to send the webpage to that user.
A good metaphor I read somewhere is like if the server was a waitress and the requests it is receiving are like restaurant customers. How many customers could a restaurant serve if the single waitress could only wait on one table at a time? Almost the whole time would be wasted while the waitress just sits and smokes a cigarette and watches her one table eat, all the while other customers are staring at the walls waiting for the waitress. The only way to fix that would be to hire more waitresses, who will all also sit and smoke cigarettes while their one table chooses what they want to order, eats, etc. That is the same solution as multithreading. Most of the time each thread uses is wasted waiting for I/O to be performed (some estimates put it around 98 - 99% of its time). However, the async solution would be more like restaurants actually work: Since each customer requires such a small amount of actual time interacting with the waitress, she handles many tables at a time by helping the other tables while the customers choose what to order, eat, etc.
The actual figures are staggering as to what takes what amount of time in a modern computer. Here are some figures:
CPU executes a typical operation (ADD, SUB, MOV, etc.): 1 nanosecond
CPU executes more complex operation (MUL, DIV): 2 - 3 nanoseconds
Fetch from L1 cache: 0.5 nanosecond
Fetch from L2 cache: 7 nanoseconds
Fetch from main memory: 100 nanoseconds
Send 2K bytes over 1Gbps network: 20,000 nanoseconds
Read 1MB sequentially from memory: 250,000 nanoseconds
Read 1MB sequentially from disk: 20,000,000 nanoseconds
Send packet from US to Europe and back: 150,000,000 nanoseconds
As you can see, the CPU, cache and even the main memory are orders of magnitude faster than the disk and network. So you can see that I/O is what is holding back any computer from the true speeds its processor can attain. So when you make those disk and network operations non-blocking, you are casting off that bottleneck. That is why a single-threaded node server can keep up quite well with an 8-core server running with multithreading. Because that single-thread is only dealing with the super-fast CPU, cache and main memory operations. Disk and network access and sequential memory access are all handled asynchronously since they are very slow.
tourmaline wrote:Ok, updating character positions in a game, players in game, environment changes in a game; asynchronously, that makes sense. Broadcasting to 100 clients, yes, asynchronous. But I don't see any benefit to reading a file. I don't see a benefit to making a DB query asynchronously. I always want to use the data at the very moment I am querying it; not down the road sometime. If I insert data, I don't want to let the program churn on before I've verified the insert.
To the computer the result comes back later, but it still happens so fast a human can't comprehend any pause. Remember, we are talking about nanoseconds here. And whether or not your call is async or sync, it still takes about the same amount of time to retrieve the data. The wait isn't in node doing something else. It is in the much longer wait time on I/O versus what the CPU can perform. There can be some unpredictability to what order events can occur, but that is the way event-based programming always happens. You can't tell your users what order they need to press buttons in, or who needs to request a page first, so it doesn't really matter to event-based applications. And that tends to be how networked servers operate anyway, async or multithreaded.
tourmaline wrote:As I read, I'm finding hints about how node might work: single pipe that all clients connect to and just beats on. Ok, so having node at the end of the pipe chucking requests to where they go, all at the same time, sure. But in the end, how is that better than a separate pipe for each client? It seems like a problem for the sake of a solution.
Because having a separate pipe for each client requires multithreading, and everyone pretty much agrees multithreading sucks. That is because to multithread, you are bowing down to the almighty power of our lord "The Thread Scheduler". The thread scheduler is completely controlled by the OS. It decides when to switch threads, which one gets executed when, etc. And because of that, debugging multithreaded servers is an incredible pain. An error may only occur 1 every 100 times, or only when a completely unrelated program is running in the background at the same time, or only when a certain number of threads are executing. The thread scheduler is an invisible and untouchable, yet highly important part of your application. Once you have dozens of threads interacting, when something breaks, put on some coffee and prepare to pull your hair out.
tourmaline wrote:I've been through the manual quite a bit, and I'm finding some pretty neat things. But in practice, I'm stuck on fs.readFIle, a callback, and how that is practical to me (or anyone for that matter). I keep trying to take that example, and expand on it to how I would use it, and it just fails.
Honestly, I have only had to read a file once or twice. I would suggest to get ExpressJS from npm, set up a project and do some tutorials in that. Everything would likely make a lot more sense in that environment. That is how I got the hang of node. Then add Socket.io, and try making a simple web-based chat room. That is exactly the order I went in.
But as far as what you mentioned, fs.readFile is an async method to read an entire file. The callback function you define for the async call will receive the contents of the file as one of its parameters. I have only seen it used for writing web servers to get HTML files, which you likely won't be doing much (most good web frameworks use Connect as web server middleware).
tourmaline wrote:EXAMPLE: Instead of one file to read, read 3. give each file result back to it's calling function where it appends them to a local variable. Now give that result back to an initial function that initially decided to grab those files or not (among other functionality), based on the presence of that readfile string being present, or a set bool. It doesn't need to be 3 functions; but if I want to separate string cleaning and concat away from the logic of deciding to read the file, and a 3rd function that just performs the file read. 3 functions to get there is not completely far fetched. It's far better than a 3 level callback nest, if I could even think through such a thing.
Nested callbacks are entirely common and accepted in node. Trying to resist them leads you to some weird patterns, or, if you really don't like nested callbacks, to the
"promises" pattern/libraries. The latter is how C# handles asynchronous programming.
tourmaline wrote:I suppose I should look at something else for a minute, maybe start looking at WEBGL so I have an interface to apply things to. Maybe build a log file scheme with that. Maybe some of this node beginner stuff without input is a drag (thanks Halls for helping me notice a little bit outside of where I'm stuck).
I highly suggest to start off with Express and Socket.io. That will likely be the skeleton of your server when you move on to working on a WebGL online game anyway. Things are much clearer when working with the web to start off with. Perhaps just trying to do it with no output is what is really causing you problems, kind of like trying to learn practical PHP from only the command line tool.
Hope this helps!
