Node js script doesn't run in another folder

[peterchan75]

Hi,
The node js script is running OK on my desktop. But it throw an error when I run it in another folder.
Error message from command line.
node:internal/fs/utils:344
throw err;
^

Error: ENOENT: no such file or directory

Any idea how to fix this problem?
Thanks in advance.

 

[davidktw]

Hi,
The node js script is running OK on my desktop. But it throw an error when I run it in another folder.
Error message from command line.
node:internal/fs/utils:344
throw err;
^

Error: ENOENT: no such file or directory

Any idea how to fix this problem?
Thanks in advance.

Without looking at your codes, I would say you are reading off a file which is not relative to your process’s working directory. It can be a property file or resource or anything that your project requires or library required.

You have to first find out which file/directory is the one mentioned in your error message.
It might not be a file, it could also be a directory structure missing that is leading to a file you need.

 

[Trader11]

Without looking at your codes, I would say you are reading off a file which is not relative to your process’s working directory. It can be a property file or resource or anything that your project requires or library required.

You have to first find out which file/directory is the one mentioned in your error message.
It might not be a file, it could also be a directory structure missing that is leading to a file you need.

I think Peter hardcoded some paths....common issue

 

[davidktw]

I think Peter hardcoded some paths....common issue

If it is hard coded absolute paths, it should continue to work unless it is relative paths.

 

[peterchan75]

@david, Trader11,
Thanks.
I think bite off more that I can chew.
No CNY visit so spent past few days on learning Node js code.
I think this is relative path right ?
const currentPath = process.cwd();
Just want to try asynchronous and await. It's fast but a bit unreliable.
It's a simple programme to download json data from Yahoo finance.
It takes less than 2 second for 4000+ ticker symbols vs 40+ seconds in perl.

 

[davidktw]

@david, Trader11,
Thanks.
I think bite off more that I can chew.
No CNY visit so spent past few days on learning Node js code.
I think this is relative path right ?
const currentPath = process.cwd();
Just want to try asynchronous and await. It's fast but a bit unreliable.
It's a simple programme to download json data from Yahoo finance.
It takes less than 2 second for 4000+ ticker symbols vs 40+ seconds in perl.

Nope process.cwd() should give you an absolute directory path. You sure that is the reason for that error?

You sure the slowness is a Perl problem?
https://metacpan.org/pod/Future::AsyncAwait

 

[peterchan75]

Nope process.cwd() should give you an absolute directory path. You sure that is the reason for that error?

You sure the slowness is a Perl problem?
https://metacpan.org/pod/Future::AsyncAwait

I never use asynchronous function in perl. Don't need to.
Is this correct way to specify a path in Note js ?
const output_eod = 'C:\\Users\\Peter\\Desktop\\yahoo_api_eod.txt';

 

[davidktw]

I never use asynchronous function in perl. Don't need to.
Is this correct way to specify a path in Note js ?
const output_eod = 'C:\\Users\\Peter\\Desktop\\yahoo_api_eod.txt';

It should work if the missing file is your problem and your file can be found at that path.

 

[peterchan75]

I found the problem. Since asynchronous is all about timing, I add some delay and that fix the problem.
for (i=0; i <= 10000; i++) {} Node js may have a function to do that. Too lazy to go and search for it.
I learn this since Fortran time.
The error happened when I tried to reopen the file for verification. It was not ready.

Further update....
That delay didn't fix the problem. It was during my debugging that I commented out portion of the code that did verification and run the program. After that I can uncomment the verification code and it runs fine. I can move the program to any place. My comment and uncomment seems to fix the problem. It's repeatable. Any hidden setup or init files?

 

[davidktw]

I found the problem. Since asynchronous is all about timing, I add some delay and that fix the problem.
for (i=0; i <= 10000; i++) {} Node js may have a function to do that. Too lazy to go and search for it.
I learn this since Fortran time.
The error happened when I tried to reopen the file for verification. It was not ready.

If you need to introduce this kind of delay in an asynchronous design, you are probably doing it wrong. In such scenarios, you should be using event driven approach that normally comes with deferred/future/promise design that uses callback to be notified of events. You are not working in low-level scenarios such as electrical-electronics to warrant the use of deliberate delays.

Further update....
That delay didn't fix the problem. It was during my debugging that I commented out portion of the code that did verification and run the program. After that I can uncomment the verification code and it runs fine. I can move the program to any place. My comment and uncomment seems to fix the problem. It's repeatable. Any hidden setup or init files?

Can't see what is your problem unless I can see what does (un)?commenting your codes does and whether it has any relationship with the above timing issue that timing situation has changed because of (not)? running your vertification codes.

 

[peterchan75]

If you need to introduce this kind of delay in an asynchronous design, you are probably doing it wrong. In such scenarios, you should be using event driven approach that normally comes with deferred/future/promise design that uses callback to be notified of events. You are not working in low-level scenarios such as electrical-electronics to warrant the use of deliberate delays.


Can't see what is your problem unless I can see what does (un)?commenting your codes does and whether it has any relationship with the above timing issue that timing situation has changed because of (not)? running your vertification codes.

Hi David,
I think I have "fixed" the issue. By commenting out some of the codes, files that are not already in the directory will be created. Otherwise, it will give an error either file or directory does not exist. Once, the files are in the directory, the program works fine. So I put this line at the beginning and it fixes the problem.
fs.closeSync(fs.openSync(output_eod, 'w'));

 

[davidktw]

Hi David,
I think I have "fixed" the issue. By commenting out some of the codes, files that are not already in the directory will be created. Otherwise, it will give an error either file or directory does not exist. Once, the files are in the directory, the program works fine. So I put this line at the beginning and it fixes the problem.
fs.closeSync(fs.openSync(output_eod, 'w'));

Well you know better, I won’t since I wouldn’t know if those files there or not there has significance or not.

Good for you if you have fixed your problem.

 

[peterchan75]

Well you know better, I won’t since I wouldn’t know if those files there or not there has significance or not.

Good for you if you have fixed your problem.

David,
Does this mean that fs.createWriteStream is asynchronous ?
I have to use the earlier code to force open an empty file.
As for the files, all are text files. 2 are for text data from https request and 1 for text input.
fs.readFileSync for reading and fs.createWriteStream for writing.
There is no issue with reading. For writing, if files do not already exist, must use fs.closeSync(fs.openSync(output_eod, 'w')); to force open.

 

[davidktw]

David,
Does this mean that fs.createWriteStream is asynchronous ?
I have to use the earlier code to force open an empty file.
As for the files, all are text files. 2 are for text data from https request and 1 for text input.
fs.readFileSync for reading and fs.createWriteStream for writing.
There is no issue with reading. For writing, if files do not already exist, must use fs.closeSync(fs.openSync(output_eod, 'w')); to force open.

createWriteStream is an async function that return Promise.
It is under the Promise API, which you can refer to https://nodejs.org/api/fs.html#filehandlecreatewritestreamoptions for the actual documentation.
You can also refer to https://nodejs.org/en/knowledge/advanced/streams/how-to-use-fs-create-write-stream/

If you have no need for concurrent operations, you should use the synchronous fs operations unless you are willing to take care of the async behaviour.

I don't do Node.JS often enough. Here is an example of how you want to read after write. I purposely simplify it by using a synchronous read operation since what is important here is to demonstrate the read must come after write.

var fs = require('fs');

function writeFile() {
  return new Promise(resolve => {
    const stream = fs.createWriteStream("/tmp/hello");
    stream.write(new Date() + "\n");
    stream.close();
    stream.on('finish', resolve);
  });
}

(async () => {
  let t = writeFile();
  console.log("PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:");
  console.log(t);

  t.then(() => {
    console.log("PROMISE STATE AFTER PROMISE IS RESOLVED:");
    console.log(t);

    const data = fs.readFileSync('/tmp/hello', {
      encoding:'utf8', flag:'r'
    });
    // Display the file data
    console.log("READ DATA: " + data);
  });
})();

$ node testfs.js
PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:
Promise { <pending> }
PROMISE STATE AFTER PROMISE IS RESOLVED:
Promise { undefined }
READ DATA: Sat Feb 05 2022 15:38:13 GMT+0800 (Singapore Standard Time)
$

If you comment the stream.on('finish', resolve); line and execute it, the read operation may execute before the write complete and you will get this result when executed

$ node testfs.js
PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:
Promise { <pending> }
$

There is another example for simplicity

var fs = require('fs');

const stream = fs.createWriteStream("/tmp/hello");
stream.write(new Date() + "\n");
stream.close();
stream.on('finish', () => {
  const data = fs.readFileSync('/tmp/hello', {
    encoding:'utf8', flag:'r'
  });
  // Display the file data
  console.log("READ DATA: " + data);
});

$ node testfs2.js
READ DATA: Sat Feb 05 2022 15:42:20 GMT+0800 (Singapore Standard Time)

$

 

[peterchan75]

createWriteStream is an async function that return Promise.
It is under the Promise API, which you can refer to https://nodejs.org/api/fs.html#filehandlecreatewritestreamoptions for the actual documentation.
You can also refer to https://nodejs.org/en/knowledge/advanced/streams/how-to-use-fs-create-write-stream/

If you have no need for concurrent operations, you should use the synchronous fs operations unless you are willing to take care of the async behaviour.

I don't do Node.JS often enough. Here is an example of how you want to read after write. I purposely simplify it by using a synchronous read operation since what is important here is to demonstrate the read must come after write.

var fs = require('fs');

function writeFile() {
  return new Promise(resolve => {
    const stream = fs.createWriteStream("/tmp/hello");
    stream.write(new Date() + "\n");
    stream.close();
    stream.on('finish', resolve);
  });
}

(async () => {
  let t = writeFile();
  console.log("PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:");
  console.log(t);

  t.then(() => {
    console.log("PROMISE STATE AFTER PROMISE IS RESOLVED:");
    console.log(t);

    const data = fs.readFileSync('/tmp/hello', {
      encoding:'utf8', flag:'r'
    });
    // Display the file data
    console.log("READ DATA: " + data);
  });
})();

$ node testfs.js
PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:
Promise { <pending> }
PROMISE STATE AFTER PROMISE IS RESOLVED:
Promise { undefined }
READ DATA: Sat Feb 05 2022 15:38:13 GMT+0800 (Singapore Standard Time)
$

If you comment the stream.on('finish', resolve); line and execute it, the read operation may execute before the write complete and you will get this result when executed

$ node testfs.js
PROMISE STATE IMMEDIATELY AFTER INVOKING writeFile:
Promise { <pending> }
$

Thanks.
Wasted 5 days wrestling with this asynchronous thing. The result looks funny too i.e. out of order unlike synchronous. Using call centre as an analogy from stackoverflow, synchronous is to call and hang on until getting answer. Where as asynchronous is to call and hang up and wait for call back. To me, the flow is messy and hard to debug.

 

[davidktw]

Thanks.
Wasted 5 days wrestling with this asynchronous thing. The result looks funny too i.e. out of order unlike synchronous. Using call centre as an analogy from stackoverflow, synchronous is to call and hang on until getting answer. Where as asynchronous is to call and hang up and wait for call back. To me, the flow is messy and hard to debug.

That is why async/await is introduced to create the illusion of a synchronous flow of code instructions.

But still depending on how some APIs are designed, you may need to provide your own wrapping around APIs that still uses Promise style or the much older callback style.

I don’t do native node.js nor javascript as my core competency, so maybe other developers than do JS day in day out may be able to provide more insightful opinions.

 

[davidktw]

@peterchan75
Extra reading for you
https://www.freecodecamp.org/news/node-js-streams-everything-you-need-to-know-c9141306be93/
Here is another simple code for you to play with

$ cat readwritefiledemo.js
var fs = require('fs');

const fspromises = fs.promises;
const fd = fspromises.open('./demo.txt', 'r');

console.log('INFO: Going to read ./demo.txt');
fd.then(async (fh) => {

  let data = await fh.readFile("utf-8");
  console.log(`INFO: ${data}`);
  await fh.close;
}).catch((error) => {
  console.log(`ERROR: ${error}`);
  console.log('INFO Going to create ./demo.txt');

  const fd2 = fspromises.open('./demo.txt', 'w');
  fd2.then(async (fh) => {
    console.log('INFO Going to write the current date into ./demo.txt');
    await fh.writeFile(new Date() + "\n", "utf-8");
    await fh.close;
  }).catch((error) => {
    console.log(`ERROR: ${error}`);
  });
});

Running the code

$ rm ./demo.txt
rm: ./demo.txt: No such file or directory
$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
ERROR: Error: ENOENT: no such file or directory, open './demo.txt'
INFO Going to create ./demo.txt
INFO Going to write the current date into ./demo.txt
$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
INFO: Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)

$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
INFO: Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)

$ cat ./demo.txt
Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)
$

 

[peterchan75]

@peterchan75
Extra reading for you
https://www.freecodecamp.org/news/node-js-streams-everything-you-need-to-know-c9141306be93/
Here is another simple code for you to play with

$ cat readwritefiledemo.js
var fs = require('fs');

const fspromises = fs.promises;
const fd = fspromises.open('./demo.txt', 'r');

console.log('INFO: Going to read ./demo.txt');
fd.then(async (fh) => {

  let data = await fh.readFile("utf-8");
  console.log(`INFO: ${data}`);
  await fh.close;
}).catch((error) => {
  console.log(`ERROR: ${error}`);
  console.log('INFO Going to create ./demo.txt');

  const fd2 = fspromises.open('./demo.txt', 'w');
  fd2.then(async (fh) => {
    console.log('INFO Going to write the current date into ./demo.txt');
    await fh.writeFile(new Date() + "\n", "utf-8");
    await fh.close;
  }).catch((error) => {
    console.log(`ERROR: ${error}`);
  });
});

Running the code

$ rm ./demo.txt
rm: ./demo.txt: No such file or directory
$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
ERROR: Error: ENOENT: no such file or directory, open './demo.txt'
INFO Going to create ./demo.txt
INFO Going to write the current date into ./demo.txt
$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
INFO: Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)

$ node readwritefiledemo.js
INFO: Going to read ./demo.txt
INFO: Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)

$ cat ./demo.txt
Sun Feb 06 2022 09:54:17 GMT+0800 (Singapore Standard Time)
$

Hi Divid,
I am still wrestling with the file from asynchronous function this morning. It was OK yesterday. I try to read back and verify its content in the same program. I have changed all file writing to synchronous mode. The file isn't there. I move the verification code to a different script and it works like a charm.

Thanks for the reading material.
Maybe after reading this material I can fix this particular problem.

 

[davidktw]

Hi Divid,
I am still wrestling with the file from asynchronous function this morning. It was OK yesterday. I try to read back and verify its content in the same program. I have changed all file writing to synchronous mode. The file isn't there. I move the verification code to a different script and it works like a charm.

Thanks for the reading material.
Maybe after reading this material I can fix this particular problem.

Without looking at your code, I wouldn’t be able to properly help.

Hence you will need to figure that out yourself. I have already show you some ways to use the asynchronous approach to read files. But unless you need that kind of performance even at initial stage of your code, you should just use the synchronous api first.

 

[peterchan75]

Without looking at your code, I wouldn’t be able to properly help.

Hence you will need to figure that out yourself. I have already show you some ways to use the asynchronous approach to read files. But unless you need that kind of performance even at initial stage of your code, you should just use the synchronous api first.

Just need to run 2 scripts. One for download and one for verification.