std/archive/mod.ts
Doctor 9298ea503f
feat(archive): UntarStream and TarStream (#4548)
* refactor(archive): An implementation of Tar as streams

* fmt(archive): Ran `deno fmt`

* fix(archive): fixed JSDoc examples in tar_streams.ts

* fix(archive): fixed JSDoc examples so `deno task test` doesn't complain

* fix(archive): lint license error

* fix(archive): lint error files not exported

* set(archive): Set current time as mtime for default

* resolve(archive): resolves comments made

* add(archive): `{ mode: 'byob' }` support for TarStream

* add(archive): `{ mode: 'byob' }` support for UnTarStream

* adjust(archive): The logical flow of a few if statements

* tests(archive): Updated Tests for Un/TarStream

* fix(archive): TarStream mtime wasn't an octal

* fix(archive): TarStream tests

* add(archive): Added parsePathname function

Added parsePathname function abstracting the logic out of TarStream allowing the developer to validate pathnames before providing them to TarStream hoping it doesn't throw an error and require the archive creation to start all over again.

* fix(archive): extra bytes incorrectly appending at the end of files

When the appending file was exactly divisible by 512 bytes, an extra 512 bytes was being appending instead of zero to fill in the gap, causing the next header to be read at the wrong place.

* adjust(archive): to always return the amount of bytes requested

Instead of using enqueue, the leftover bytes are saved for later for the next buffer provided.

* tweaks

* fix

* docs(archive): Link to the spec that they're following

* docs(archive): fix spelling

* add(archive): function validTarSteamOptions

- To make sure, if TarStreamOptions are being provided, that they are in the correct format so as to not create bad tarballs.

* add(archive): more tests

* fix(archive): validTarStreamOptions

* add(archive): tests for validTarStreamOptions

* refactor(archive): code to copy the changes made in the @doctor/tar-stream version

* test(archive): added from @doctor/tar-stream

* chore: nit on anonymous function

* refactor(archive): UnTarStream that fixes unexplainable memory leak

- The second newest test introduced here '... with invalid ending' seems to detect a memory leak due to an invalid tarball. I couldn't figure out why the memory leak was happening but I know this restructure of the code doesn't have that same memory leak.

* chore: fmt

* tests(archive): added remaining tests to cover many lines as possible

* adjust(archive): remove simplify pathname code

* adjust(archive): remove checking for duplicate pathnames in taring process

* adjust(archive): A readable will exist on TarEntry unless string values 1-6

* tests(archive): added more tests for higher coverage

* adjust(archives): TarStream and UnTarStream to implement TransformStream

* docs(archive): moved TarStreamOptions docs into properties.

* adjust(archive): TarStreamFile to take a ReadableSteam instead of an Iterable | AsyncIterable

* adjust(archive): to use FixedChunkStream instead of rolling it's own implementation

* fix(archive): lint error

* adjust(archive): Error types and messages

* adjust(archive): more Error messages / improve tests

* refactor(archive): UnTarStream to return TarStreamChunk instead of TarStreamEntry

* fix(archive): JSDoc example

* adjust(archive): mode, uid, gid options to be provided as numbers instead of strings.

* adjust(archive): TarStream's pathname to be only of type string

* fix(archive): prefix/name to ignore everything past the first NULL

* adjust(archive): `checksum` and `pad` to not be exposed from UnTarStream

* adjust(archive): checksum calculation

* change(archive): `.slice` to `.subarray`

* doc(archive): "octal number" to "octal literal"

* adjust(archive): TarStreamOptions to be optional with defaults

* doc(archive): added more docs for the interfaces

* docs(archive): denoting defaults

* docs(archive): updated for new lint rules

* adjust(archive): Tests to use assertRejects where appropriate & add `validPathname` function

- The `validPathname` is meant to be a nicer exposed function for users of this lib to validate that their pathnames are valid before pipping it through the TarStream, over exposing parsePathname where the user may be confused about what to do with the result.

* adjust(archive): to use `Date.now()` instead of `new Date().getTime()`

Co-authored-by: ud2 <sjx233@qq.com>

* adjust(archive): mode, uid, and gid to be numbers instead of strings when Untaring

* tests(archive): adjust two tests to also validate the contents of the files are valid

* adjust(archive): linkname, uname, and gname to follow the same decoding rules as name and prefix

* rename(archive): UnTarStream to UntarStream

* fix(archive): type that was missed getting updated

* tests(archive): adjust check headers test to validate all header properties instead of just pathnames

* rename(archive): `pathname` properties to `path`

* docs(archive): updated to be more descriptive

* docs(archive): Updated error types

* adjust(archive): `validPath` to `assertValidPath`

* adjust(archive): `validTarStreamOptions` to `assertValidTarStreamOptions`

* revert(archive): UntarStream to produce TarStreamEntry instead of TarStreamChunk

* refactor: remove redundant `void` return types

* docs: cleanup assertion function docs

* docs: correct `TarStream` example

* docs: minor docs cleanups

* refactor: improve error class specificity

* docs: add `@experimental` JSDoc tags

* docs(archive): Updated examples for `assertValidPath` and `assertValidTarStreamOptions```

* fix(archive): problem with tests

- I suspect the problem is that a file that was read by `Deno.readDir` changed size between being read at `Deno.stat` and when `Deno.open` finished pulling it all in.

* update error messages

* update error messages

* fix typos

* refactor: tweak error messages

* refactor: tweaks and add type field

---------

Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
Co-authored-by: ud2 <sjx233@qq.com>
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
2024-09-02 17:43:22 +10:00

73 lines
2.6 KiB
TypeScript

// Copyright 2018-2024 the Deno authors. All rights reserved. MIT license.
/*!
* Ported and modified from: https://github.com/beatgammit/tar-js and
* licensed as:
*
* (The MIT License)
*
* Copyright (c) 2011 T. Jameson Little
* Copyright (c) 2019 Jun Kato
* Copyright (c) 2018-2024 the Deno authors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
/**
* Tar is a utility for collecting multiple files (or any arbitrary data) into one
* archive file, while untar is the inverse utility to extract the files from an
* archive. Files are not compressed, only collected into the archive.
*
* ```ts no-eval
* import { Tar } from "@std/archive/tar";
* import { Buffer } from "@std/io/buffer";
* import { copy } from "@std/io/copy";
*
* const tar = new Tar();
*
* // Now that we've created our tar, let's add some files to it:
*
* const content = new TextEncoder().encode("Some arbitrary content");
* await tar.append("deno.txt", {
* reader: new Buffer(content),
* contentSize: content.byteLength,
* });
*
* // This file is sourced from the filesystem (and renamed in the archive)
* await tar.append("filename_in_archive.txt", {
* filePath: "./filename_on_filesystem.txt",
* });
*
* // Now let's write the tar (with it's two files) to the filesystem
* // use tar.getReader() to read the contents.
*
* const writer = await Deno.open("./out.tar", { write: true, create: true });
* await copy(tar.getReader(), writer);
* writer.close();
* ```
*
* @experimental **UNSTABLE**: New API, yet to be vetted.
*
* @module
*/
export * from "./tar.ts";
export * from "./untar.ts";
export * from "./tar_stream.ts";
export * from "./untar_stream.ts";