Projekt

Obecné

Profil

Stáhnout (20 KB) Statistiky
| Větev: | Revize:
1
# cacache [![npm version](https://img.shields.io/npm/v/cacache.svg)](https://npm.im/cacache) [![license](https://img.shields.io/npm/l/cacache.svg)](https://npm.im/cacache) [![Travis](https://img.shields.io/travis/npm/cacache.svg)](https://travis-ci.org/npm/cacache) [![AppVeyor](https://ci.appveyor.com/api/projects/status/github/npm/cacache?svg=true)](https://ci.appveyor.com/project/npm/cacache) [![Coverage Status](https://coveralls.io/repos/github/npm/cacache/badge.svg?branch=latest)](https://coveralls.io/github/npm/cacache?branch=latest)
2

    
3
[`cacache`](https://github.com/npm/cacache) is a Node.js library for managing
4
local key and content address caches. It's really fast, really good at
5
concurrency, and it will never give you corrupted data, even if cache files
6
get corrupted or manipulated.
7

    
8
On systems that support user and group settings on files, cacache will
9
match the `uid` and `gid` values to the folder where the cache lives, even
10
when running as `root`.
11

    
12
It was written to be used as [npm](https://npm.im)'s local cache, but can
13
just as easily be used on its own.
14

    
15
_Translations: [español](README.es.md)_
16

    
17
## Install
18

    
19
`$ npm install --save cacache`
20

    
21
## Table of Contents
22

    
23
* [Example](#example)
24
* [Features](#features)
25
* [Contributing](#contributing)
26
* [API](#api)
27
  * [Using localized APIs](#localized-api)
28
  * Reading
29
    * [`ls`](#ls)
30
    * [`ls.stream`](#ls-stream)
31
    * [`get`](#get-data)
32
    * [`get.stream`](#get-stream)
33
    * [`get.info`](#get-info)
34
    * [`get.hasContent`](#get-hasContent)
35
  * Writing
36
    * [`put`](#put-data)
37
    * [`put.stream`](#put-stream)
38
    * [`put*` opts](#put-options)
39
    * [`rm.all`](#rm-all)
40
    * [`rm.entry`](#rm-entry)
41
    * [`rm.content`](#rm-content)
42
  * Utilities
43
    * [`setLocale`](#set-locale)
44
    * [`clearMemoized`](#clear-memoized)
45
    * [`tmp.mkdir`](#tmp-mkdir)
46
    * [`tmp.withTmp`](#with-tmp)
47
  * Integrity
48
    * [Subresource Integrity](#integrity)
49
    * [`verify`](#verify)
50
    * [`verify.lastRun`](#verify-last-run)
51

    
52
### Example
53

    
54
```javascript
55
const cacache = require('cacache/en')
56
const fs = require('fs')
57

    
58
const tarball = '/path/to/mytar.tgz'
59
const cachePath = '/tmp/my-toy-cache'
60
const key = 'my-unique-key-1234'
61

    
62
// Cache it! Use `cachePath` as the root of the content cache
63
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
64
  console.log(`Saved content to ${cachePath}.`)
65
})
66

    
67
const destination = '/tmp/mytar.tgz'
68

    
69
// Copy the contents out of the cache and into their destination!
70
// But this time, use stream instead!
71
cacache.get.stream(
72
  cachePath, key
73
).pipe(
74
  fs.createWriteStream(destination)
75
).on('finish', () => {
76
  console.log('done extracting!')
77
})
78

    
79
// The same thing, but skip the key index.
80
cacache.get.byDigest(cachePath, integrityHash).then(data => {
81
  fs.writeFile(destination, data, err => {
82
    console.log('tarball data fetched based on its sha512sum and written out!')
83
  })
84
})
85
```
86

    
87
### Features
88

    
89
* Extraction by key or by content address (shasum, etc)
90
* [Subresource Integrity](#integrity) web standard support
91
* Multi-hash support - safely host sha1, sha512, etc, in a single cache
92
* Automatic content deduplication
93
* Fault tolerance (immune to corruption, partial writes, process races, etc)
94
* Consistency guarantees on read and write (full data verification)
95
* Lockless, high-concurrency cache access
96
* Streaming support
97
* Promise support
98
* Pretty darn fast -- sub-millisecond reads and writes including verification
99
* Arbitrary metadata storage
100
* Garbage collection and additional offline verification
101
* Thorough test coverage
102
* There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
103

    
104
### Contributing
105

    
106
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The [Contributor Guide](CONTRIBUTING.md) has all the information you need for everything from reporting bugs to contributing entire new features. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
107

    
108
All participants and maintainers in this project are expected to follow [Code of Conduct](CODE_OF_CONDUCT.md), and just generally be excellent to each other.
109

    
110
Please refer to the [Changelog](CHANGELOG.md) for project history details, too.
111

    
112
Happy hacking!
113

    
114
### API
115

    
116
#### <a name="localized-api"></a> Using localized APIs
117

    
118
cacache includes a complete API in English, with the same features as other
119
translations. To use the English API as documented in this README, use
120
`require('cacache/en')`. This is also currently the default if you do
121
`require('cacache')`, but may change in the future.
122

    
123
cacache also supports other languages! You can find the list of currently
124
supported ones by looking in `./locales` in the source directory. You can use
125
the API in that language with `require('cacache/<lang>')`.
126

    
127
Want to add support for a new language? Please go ahead! You should be able to
128
copy `./locales/en.js` and `./locales/en.json` and fill them in. Translating the
129
`README.md` is a bit more work, but also appreciated if you get around to it. 👍🏼
130

    
131
#### <a name="ls"></a> `> cacache.ls(cache) -> Promise<Object>`
132

    
133
Lists info for all entries currently in the cache as a single large object. Each
134
entry in the object will be keyed by the unique index key, with corresponding
135
[`get.info`](#get-info) objects as the values.
136

    
137
##### Example
138

    
139
```javascript
140
cacache.ls(cachePath).then(console.log)
141
// Output
142
{
143
  'my-thing': {
144
    key: 'my-thing',
145
    integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
146
    path: '.testcache/content/deadbeef', // joined with `cachePath`
147
    time: 12345698490,
148
    size: 4023948,
149
    metadata: {
150
      name: 'blah',
151
      version: '1.2.3',
152
      description: 'this was once a package but now it is my-thing'
153
    }
154
  },
155
  'other-thing': {
156
    key: 'other-thing',
157
    integrity: 'sha1-ANothER+hasH=',
158
    path: '.testcache/content/bada55',
159
    time: 11992309289,
160
    size: 111112
161
  }
162
}
163
```
164

    
165
#### <a name="ls-stream"></a> `> cacache.ls.stream(cache) -> Readable`
166

    
167
Lists info for all entries currently in the cache as a single large object.
168

    
169
This works just like [`ls`](#ls), except [`get.info`](#get-info) entries are
170
returned as `'data'` events on the returned stream.
171

    
172
##### Example
173

    
174
```javascript
175
cacache.ls.stream(cachePath).on('data', console.log)
176
// Output
177
{
178
  key: 'my-thing',
179
  integrity: 'sha512-BaSe64HaSh',
180
  path: '.testcache/content/deadbeef', // joined with `cachePath`
181
  time: 12345698490,
182
  size: 13423,
183
  metadata: {
184
    name: 'blah',
185
    version: '1.2.3',
186
    description: 'this was once a package but now it is my-thing'
187
  }
188
}
189

    
190
{
191
  key: 'other-thing',
192
  integrity: 'whirlpool-WoWSoMuchSupport',
193
  path: '.testcache/content/bada55',
194
  time: 11992309289,
195
  size: 498023984029
196
}
197

    
198
{
199
  ...
200
}
201
```
202

    
203
#### <a name="get-data"></a> `> cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})`
204

    
205
Returns an object with the cached data, digest, and metadata identified by
206
`key`. The `data` property of this object will be a `Buffer` instance that
207
presumably holds some data that means something to you. I'm sure you know what
208
to do with it! cacache just won't care.
209

    
210
`integrity` is a [Subresource
211
Integrity](#integrity)
212
string. That is, a string that can be used to verify `data`, which looks like
213
`<hash-algorithm>-<base64-integrity-hash>`.
214

    
215
If there is no content identified by `key`, or if the locally-stored data does
216
not pass the validity checksum, the promise will be rejected.
217

    
218
A sub-function, `get.byDigest` may be used for identical behavior, except lookup
219
will happen by integrity hash, bypassing the index entirely. This version of the
220
function *only* returns `data` itself, without any wrapper.
221

    
222
##### Note
223

    
224
This function loads the entire cache entry into memory before returning it. If
225
you're dealing with Very Large data, consider using [`get.stream`](#get-stream)
226
instead.
227

    
228
##### Example
229

    
230
```javascript
231
// Look up by key
232
cache.get(cachePath, 'my-thing').then(console.log)
233
// Output:
234
{
235
  metadata: {
236
    thingName: 'my'
237
  },
238
  integrity: 'sha512-BaSe64HaSh',
239
  data: Buffer#<deadbeef>,
240
  size: 9320
241
}
242

    
243
// Look up by digest
244
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
245
// Output:
246
Buffer#<deadbeef>
247
```
248

    
249
#### <a name="get-stream"></a> `> cacache.get.stream(cache, key, [opts]) -> Readable`
250

    
251
Returns a [Readable Stream](https://nodejs.org/api/stream.html#stream_readable_streams) of the cached data identified by `key`.
252

    
253
If there is no content identified by `key`, or if the locally-stored data does
254
not pass the validity checksum, an error will be emitted.
255

    
256
`metadata` and `integrity` events will be emitted before the stream closes, if
257
you need to collect that extra data about the cached entry.
258

    
259
A sub-function, `get.stream.byDigest` may be used for identical behavior,
260
except lookup will happen by integrity hash, bypassing the index entirely. This
261
version does not emit the `metadata` and `integrity` events at all.
262

    
263
##### Example
264

    
265
```javascript
266
// Look up by key
267
cache.get.stream(
268
  cachePath, 'my-thing'
269
).on('metadata', metadata => {
270
  console.log('metadata:', metadata)
271
}).on('integrity', integrity => {
272
  console.log('integrity:', integrity)
273
}).pipe(
274
  fs.createWriteStream('./x.tgz')
275
)
276
// Outputs:
277
metadata: { ... }
278
integrity: 'sha512-SoMeDIGest+64=='
279

    
280
// Look up by digest
281
cache.get.stream.byDigest(
282
  cachePath, 'sha512-SoMeDIGest+64=='
283
).pipe(
284
  fs.createWriteStream('./x.tgz')
285
)
286
```
287

    
288
#### <a name="get-info"></a> `> cacache.get.info(cache, key) -> Promise`
289

    
290
Looks up `key` in the cache index, returning information about the entry if
291
one exists.
292

    
293
##### Fields
294

    
295
* `key` - Key the entry was looked up under. Matches the `key` argument.
296
* `integrity` - [Subresource Integrity hash](#integrity) for the content this entry refers to.
297
* `path` - Filesystem path where content is stored, joined with `cache` argument.
298
* `time` - Timestamp the entry was first added on.
299
* `metadata` - User-assigned metadata associated with the entry/content.
300

    
301
##### Example
302

    
303
```javascript
304
cacache.get.info(cachePath, 'my-thing').then(console.log)
305

    
306
// Output
307
{
308
  key: 'my-thing',
309
  integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='
310
  path: '.testcache/content/deadbeef',
311
  time: 12345698490,
312
  size: 849234,
313
  metadata: {
314
    name: 'blah',
315
    version: '1.2.3',
316
    description: 'this was once a package but now it is my-thing'
317
  }
318
}
319
```
320

    
321
#### <a name="get-hasContent"></a> `> cacache.get.hasContent(cache, integrity) -> Promise`
322

    
323
Looks up a [Subresource Integrity hash](#integrity) in the cache. If content
324
exists for this `integrity`, it will return an object, with the specific single integrity hash
325
that was found in `sri` key, and the size of the found content as `size`. If no content exists for this integrity, it will return `false`.
326

    
327
##### Example
328

    
329
```javascript
330
cacache.get.hasContent(cachePath, 'sha256-MUSTVERIFY+ALL/THINGS==').then(console.log)
331

    
332
// Output
333
{
334
  sri: {
335
    source: 'sha256-MUSTVERIFY+ALL/THINGS==',
336
    algorithm: 'sha256',
337
    digest: 'MUSTVERIFY+ALL/THINGS==',
338
    options: []
339
  },
340
  size: 9001
341
}
342

    
343
cacache.get.hasContent(cachePath, 'sha521-NOT+IN/CACHE==').then(console.log)
344

    
345
// Output
346
false
347
```
348

    
349
#### <a name="put-data"></a> `> cacache.put(cache, key, data, [opts]) -> Promise`
350

    
351
Inserts data passed to it into the cache. The returned Promise resolves with a
352
digest (generated according to [`opts.algorithms`](#optsalgorithms)) after the
353
cache entry has been successfully written.
354

    
355
##### Example
356

    
357
```javascript
358
fetch(
359
  'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
360
).then(data => {
361
  return cacache.put(cachePath, 'registry.npmjs.org|cacache@1.0.0', data)
362
}).then(integrity => {
363
  console.log('integrity hash is', integrity)
364
})
365
```
366

    
367
#### <a name="put-stream"></a> `> cacache.put.stream(cache, key, [opts]) -> Writable`
368

    
369
Returns a [Writable
370
Stream](https://nodejs.org/api/stream.html#stream_writable_streams) that inserts
371
data written to it into the cache. Emits an `integrity` event with the digest of
372
written contents when it succeeds.
373

    
374
##### Example
375

    
376
```javascript
377
request.get(
378
  'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
379
).pipe(
380
  cacache.put.stream(
381
    cachePath, 'registry.npmjs.org|cacache@1.0.0'
382
  ).on('integrity', d => console.log(`integrity digest is ${d}`))
383
)
384
```
385

    
386
#### <a name="put-options"></a> `> cacache.put options`
387

    
388
`cacache.put` functions have a number of options in common.
389

    
390
##### `opts.metadata`
391

    
392
Arbitrary metadata to be attached to the inserted key.
393

    
394
##### `opts.size`
395

    
396
If provided, the data stream will be verified to check that enough data was
397
passed through. If there's more or less data than expected, insertion will fail
398
with an `EBADSIZE` error.
399

    
400
##### `opts.integrity`
401

    
402
If present, the pre-calculated digest for the inserted content. If this option
403
if provided and does not match the post-insertion digest, insertion will fail
404
with an `EINTEGRITY` error.
405

    
406
`algorithms` has no effect if this option is present.
407

    
408
##### `opts.algorithms`
409

    
410
Default: ['sha512']
411

    
412
Hashing algorithms to use when calculating the [subresource integrity
413
digest](#integrity)
414
for inserted data. Can use any algorithm listed in `crypto.getHashes()` or
415
`'omakase'`/`'お任せします'` to pick a random hash algorithm on each insertion. You
416
may also use any anagram of `'modnar'` to use this feature.
417

    
418
Currently only supports one algorithm at a time (i.e., an array length of
419
exactly `1`). Has no effect if `opts.integrity` is present.
420

    
421
##### `opts.memoize`
422

    
423
Default: null
424

    
425
If provided, cacache will memoize the given cache insertion in memory, bypassing
426
any filesystem checks for that key or digest in future cache fetches. Nothing
427
will be written to the in-memory cache unless this option is explicitly truthy.
428

    
429
If `opts.memoize` is an object or a `Map`-like (that is, an object with `get`
430
and `set` methods), it will be written to instead of the global memoization
431
cache.
432

    
433
Reading from disk data can be forced by explicitly passing `memoize: false` to
434
the reader functions, but their default will be to read from memory.
435

    
436
#### <a name="rm-all"></a> `> cacache.rm.all(cache) -> Promise`
437

    
438
Clears the entire cache. Mainly by blowing away the cache directory itself.
439

    
440
##### Example
441

    
442
```javascript
443
cacache.rm.all(cachePath).then(() => {
444
  console.log('THE APOCALYPSE IS UPON US 😱')
445
})
446
```
447

    
448
#### <a name="rm-entry"></a> `> cacache.rm.entry(cache, key) -> Promise`
449

    
450
Alias: `cacache.rm`
451

    
452
Removes the index entry for `key`. Content will still be accessible if
453
requested directly by content address ([`get.stream.byDigest`](#get-stream)).
454

    
455
To remove the content itself (which might still be used by other entries), use
456
[`rm.content`](#rm-content). Or, to safely vacuum any unused content, use
457
[`verify`](#verify).
458

    
459
##### Example
460

    
461
```javascript
462
cacache.rm.entry(cachePath, 'my-thing').then(() => {
463
  console.log('I did not like it anyway')
464
})
465
```
466

    
467
#### <a name="rm-content"></a> `> cacache.rm.content(cache, integrity) -> Promise`
468

    
469
Removes the content identified by `integrity`. Any index entries referring to it
470
will not be usable again until the content is re-added to the cache with an
471
identical digest.
472

    
473
##### Example
474

    
475
```javascript
476
cacache.rm.content(cachePath, 'sha512-SoMeDIGest/IN+BaSE64==').then(() => {
477
  console.log('data for my-thing is gone!')
478
})
479
```
480

    
481
#### <a name="set-locale"></a> `> cacache.setLocale(locale)`
482

    
483
Configure the language/locale used for messages and errors coming from cacache.
484
The list of available locales is in the `./locales` directory in the project
485
root.
486

    
487
_Interested in contributing more languages! [Submit a PR](CONTRIBUTING.md)!_
488

    
489
#### <a name="clear-memoized"></a> `> cacache.clearMemoized()`
490

    
491
Completely resets the in-memory entry cache.
492

    
493
#### <a name="tmp-mkdir"></a> `> tmp.mkdir(cache, opts) -> Promise<Path>`
494

    
495
Returns a unique temporary directory inside the cache's `tmp` dir. This
496
directory will use the same safe user assignment that all the other stuff use.
497

    
498
Once the directory is made, it's the user's responsibility that all files
499
within are given the appropriate `gid`/`uid` ownership settings to match
500
the rest of the cache. If not, you can ask cacache to do it for you by
501
calling [`tmp.fix()`](#tmp-fix), which will fix all tmp directory
502
permissions.
503

    
504
If you want automatic cleanup of this directory, use
505
[`tmp.withTmp()`](#with-tpm)
506

    
507
##### Example
508

    
509
```javascript
510
cacache.tmp.mkdir(cache).then(dir => {
511
  fs.writeFile(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
512
})
513
```
514

    
515
#### <a name="tmp-fix"></a> `> tmp.fix(cache) -> Promise`
516

    
517
Sets the `uid` and `gid` properties on all files and folders within the tmp
518
folder to match the rest of the cache.
519

    
520
Use this after manually writing files into [`tmp.mkdir`](#tmp-mkdir) or
521
[`tmp.withTmp`](#with-tmp).
522

    
523
##### Example
524

    
525
```javascript
526
cacache.tmp.mkdir(cache).then(dir => {
527
  writeFile(path.join(dir, 'file'), someData).then(() => {
528
    // make sure we didn't just put a root-owned file in the cache
529
    cacache.tmp.fix().then(() => {
530
      // all uids and gids match now
531
    })
532
  })
533
})
534
```
535

    
536
#### <a name="with-tmp"></a> `> tmp.withTmp(cache, opts, cb) -> Promise`
537

    
538
Creates a temporary directory with [`tmp.mkdir()`](#tmp-mkdir) and calls `cb`
539
with it. The created temporary directory will be removed when the return value
540
of `cb()` resolves -- that is, if you return a Promise from `cb()`, the tmp
541
directory will be automatically deleted once that promise completes.
542

    
543
The same caveats apply when it comes to managing permissions for the tmp dir's
544
contents.
545

    
546
##### Example
547

    
548
```javascript
549
cacache.tmp.withTmp(cache, dir => {
550
  return fs.writeFileAsync(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
551
}).then(() => {
552
  // `dir` no longer exists
553
})
554
```
555

    
556
#### <a name="integrity"></a> Subresource Integrity Digests
557

    
558
For content verification and addressing, cacache uses strings following the
559
[Subresource
560
Integrity spec](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity).
561
That is, any time cacache expects an `integrity` argument or option, it
562
should be in the format `<hashAlgorithm>-<base64-hash>`.
563

    
564
One deviation from the current spec is that cacache will support any hash
565
algorithms supported by the underlying Node.js process. You can use
566
`crypto.getHashes()` to see which ones you can use.
567

    
568
##### Generating Digests Yourself
569

    
570
If you have an existing content shasum, they are generally formatted as a
571
hexadecimal string (that is, a sha1 would look like:
572
`5f5513f8822fdbe5145af33b64d8d970dcf95c6e`). In order to be compatible with
573
cacache, you'll need to convert this to an equivalent subresource integrity
574
string. For this example, the corresponding hash would be:
575
`sha1-X1UT+IIv2+UUWvM7ZNjZcNz5XG4=`.
576

    
577
If you want to generate an integrity string yourself for existing data, you can
578
use something like this:
579

    
580
```javascript
581
const crypto = require('crypto')
582
const hashAlgorithm = 'sha512'
583
const data = 'foobarbaz'
584

    
585
const integrity = (
586
  hashAlgorithm +
587
  '-' +
588
  crypto.createHash(hashAlgorithm).update(data).digest('base64')
589
)
590
```
591

    
592
You can also use [`ssri`](https://npm.im/ssri) to have a richer set of functionality
593
around SRI strings, including generation, parsing, and translating from existing
594
hex-formatted strings.
595

    
596
#### <a name="verify"></a> `> cacache.verify(cache, opts) -> Promise`
597

    
598
Checks out and fixes up your cache:
599

    
600
* Cleans up corrupted or invalid index entries.
601
* Custom entry filtering options.
602
* Garbage collects any content entries not referenced by the index.
603
* Checks integrity for all content entries and removes invalid content.
604
* Fixes cache ownership.
605
* Removes the `tmp` directory in the cache and all its contents.
606

    
607
When it's done, it'll return an object with various stats about the verification
608
process, including amount of storage reclaimed, number of valid entries, number
609
of entries removed, etc.
610

    
611
##### Options
612

    
613
* `opts.filter` - receives a formatted entry. Return false to remove it.
614
                  Note: might be called more than once on the same entry.
615

    
616
##### Example
617

    
618
```sh
619
echo somegarbage >> $CACHEPATH/content/deadbeef
620
```
621

    
622
```javascript
623
cacache.verify(cachePath).then(stats => {
624
  // deadbeef collected, because of invalid checksum.
625
  console.log('cache is much nicer now! stats:', stats)
626
})
627
```
628

    
629
#### <a name="verify-last-run"></a> `> cacache.verify.lastRun(cache) -> Promise`
630

    
631
Returns a `Date` representing the last time `cacache.verify` was run on `cache`.
632

    
633
##### Example
634

    
635
```javascript
636
cacache.verify(cachePath).then(() => {
637
  cacache.verify.lastRun(cachePath).then(lastTime => {
638
    console.log('cacache.verify was last called on' + lastTime)
639
  })
640
})
641
```
(4-4/13)