1

So after a long day at work, I sit down and see an alert for Windows SkyDrive in the system tray:

Files can't be uploaded because the path of this file or folder is too long. Move the item to a different location or shorten its name.

C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\grunt-contrib-nodeunit\node_modules\nodeunit\node_modules\tap\node_modules\runforcover\node_modules\bunker\node_modules\burrito\node_modules\traverse\example\stringify.js

... and for a while, I laughed at that technological limitation.

But then, I wondered: is that amount of directory recursion within a Node project really necessary? It would appear that the paths beyond "angular-app\server\node_modules" are simply dependencies of the project as a whole and might be better expressed as:

C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\grunt-contrib-nodeunit\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\nodeunit\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\tap\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\runforcover\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\bunker\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\burrito\
C:\Users\Matthew\SkyDrive\Documents\Projects\Programming\angular-app\server\node_modules\traverse\

I hadn't really given it much thought before, as package management in Node seems like magic compared to many platforms.

I would imagine that some large-scale Node.js projects even contain many duplicate modules (having the same or similar versions) which could be consolidated into a lesser amount. It could be argued that:

  • The increased amount of data stored and transmitted as a result of duplicate dependencies adds to the cost of developing software.

  • Shallower directory structures (especially in this context) are often easier to navigate and understand.

  • Excessively long path names can cause problems in some computing environments.

What I am proposing (if such a thing does not exist) is a Node module that:

  • Recursively scans a Node project, collecting a list of the nested node_modules folders and how deeply they are buried in relation to the root of the project.

  • Moves the contents of each nested node_modules folder to the main node_modules folder, editing the require() calls of each .js file such that no references are broken.

  • Handles multiple versions of duplicate dependencies

If nothing else, it would make for an interesting experiment. What do you guys think? What potential problems might I encounter?

mjhasbach
  • 906
  • 9
  • 18

2 Answers2

1

See if

npm dedupe

sets you right.

API Doc here

barry-johnson
  • 3,204
  • 1
  • 17
  • 19
  • I ran that command in the server directory of [this project](https://github.com/angular-app/angular-app/tree/master/server) and was still able to navigate to that really long ```C:\...\example\``` path above. [Here](http://pastebin.com/u6UJfkuc) was the console output (some conflicts). – mjhasbach Mar 03 '14 at 04:01
  • Unfortunate. Worth a try. You can find some other ideas for this on [this SO question](http://stackoverflow.com/questions/13318364/how-to-deploy-node-js-application-with-deep-node-modules-structure-on-windows) and I feel like I saw another one on the subject a while back. Will post if I find it again. – barry-johnson Mar 03 '14 at 04:12
  • Thanks anyway for the answer (I was not aware of ```npm dedupe```) and that SO link. It sounds like @blesh had a similar idea to what I was suggesting above. – mjhasbach Mar 03 '14 at 04:20
  • You're welcome, although disappointing it couldn't solve your issue. FWIW, I think better practices by module authors in properly specifying dependencies would help. The good ones do, the bad ones just have the "1.2.3" syntax which is fairly irritating, as is the sometimes reflexive reliance on libraries, such as installing underscore just to use `_.extend()`. – barry-johnson Mar 03 '14 at 04:34
0

See fenestrate, npm-flatten, flatten-packages, npm dedupe, and multi-stage-installs.

Quoting Sam Mikes from this StackOverflow question:

npm will add dedupe-at-install-time by default. This is significantly more feasible than Node's module system changing, but it is still not exactly trivial, and involves a lot of reworking of some long-entrenched patterns.

This is (finally) currently in the works at npm, going by the name multi-stage-install, and is targeted for npm@3. npm development lead Forrest Norvell is going to spend some time running on Windows in the new year, so please do create windows-related issues on the npm issue tracker < https://github.com/npm/npm/issues >

Community
  • 1
  • 1
mjhasbach
  • 906
  • 9
  • 18