localDataStorage

localDataStorage

localDataStorage is a javascript interface for the HTML5 localStorage API that-- 1) transparently sets/gets key values using data "types" such as Array, Boolean, Date, Float, Integer, Null, Object and String; 2) provides lightweight data obfuscation; 3) intelligently compresses strings; 4) facilitates query by key (name), query by (key) value and query by existence; and 5) enforces segmented shared storage within the same domain by prefixing keys.

 
Version 1.2.0
Author: William McMeans
Date: 19 JUN 2017
 

Script tag usage

<script src="localDataStorage-1.2.0.min.js"></script>

Application:

Primary usage is the ability to seamlessly set/get keys for typically-used data types. While it's trivial to perform conversion, having it handled by the storage interface itself is exceptionally convenient. Javascript supports several primitives, and extending them into localStorage seemed a logical step. Tracking them requires 2 bytes of memory overhead, per key value.

Key values may be obfuscated using safeset/safeget. A master scramble key may be set globally, or individual scramble keys may be used per each safeset/safeget call. Scramble keys can be any value, and of any type (array, boolean, date, float, integer, etc.) Key values that have been safeset with an individual scramble key can always be retrieved, but cannot be reconstructed apart from the same individual scramble key with which they were obfuscated. The global scramble key is stored in the interface as a convenience, but individual scramble keys are not. The global scramble key may be accessed using setscramblekey/getscramblekey methods. Scrambling is not encryption. For example, no attempt is made to conceal data lengths by artificially padding to a minimum amount. This would be counter-productive to minimizing memory usage.

Strings are intelligently compressed on-the-fly. This means they are first analyzed to determine whether compression would lower the actual byte count in storage, and if so, are silently compressed/decompressed. This works well for common English texts (short-length, 7-bit ASCII), and not much else.

One may query by key name (to get the key's value), or query by value (to get any matching key names) using showkey, or query by existence using haskey. Stored values can be checked for duplicates. There are methods to prevent writing over an existing key (softset), and for deleting a key immediately upon retrieval (chopget). Memory usage can be analyzed against key values and key names, and key values can be checked for their data type. Lastly, bypass methods (forceset/forceget) permit accessing localStorage directly.

Since HTML5 localStorage is accessible to all processes running in the browser for the domain visited, it is advisable to have an interface that segments access, as much as possible. To that end, the use of prefixed keys is strongly encouraged, and localDataStorage will only read/write/delete its own keys. Unlike the HTML5 API, there is no method in this interface to delete all keys in the domain, only all prefixed keys.

The domain of operation for HTML5 localStorage is specific to the protocol, host & port; and multiple instances of localDataStorage can be run against the same domain at the same time. It is emoji-friendly 😀😆😊😵, which is to say that key values and key names are multi-byte Unicode-safe.

Dependencies:

None.
 

Events

The native localStorage change event is... lacking. Per the whims of your browser, a single page isn't permitted to listen to change events from any process, even if it triggered them. Now, in the event you'd like to listen out for changes, localDataStorage will let you. localDataStorage fires an event on key value changes, such as those made by the set, safeset, chopget or remove methods. The event returns an activity timestamp and message, as well as expected details about the affected key name with its old and new values. The old and new key value data types are also reported. Code like the following gets it done:

function nowICanSeeLocalStorageChangeEvents( e ) {
    console.log(
        "subscriber: "    + e.currentTarget.nodeName + "\n" +
        "timestamp: "     + e.detail.timestamp + " (" + new Date( e.detail.timestamp ) + ")" + "\n" +
        "prefix: "        + e.detail.prefix  + "\n" +
        "message: "       + e.detail.message + "\n" +
        "method: "        + e.detail.method  + "\n" +
        "key: "           + e.detail.key     + "\n" +
        "old value: "     + e.detail.oldval  + "\n" +
        "new value: "     + e.detail.newval  + "\n" +
        "old data type: " + e.detail.oldtype + "\n" +
        "new data type: " + e.detail.newtype
    );
};
document.addEventListener(
    "localDataStorage"
    , nowICanSeeLocalStorageChangeEvents
    , false
);

 

Wiki:

An incomplete wiki is here.
 

Example usage:

Create an instance of localDataStorage using the specified key name prefix

localData = localDataStorage( 'passphrase.life' )

--> Instantiated. Prefix adds 16.00 bytes to every key name (stored using 32.00 bytes).

 
typical set/get calls (data types are respected and returned transparently)

localData.set( 'key1', 19944.25 )

localData.get( 'key1' )

--> 19944.25

localData.set( 'key2', 2519944 )

localData.get( 'key2' )

--> 2519944

localData.set( 'key3', true )

localData.get( 'key3' )

--> true

localData.set( 'key4', 'data' )

localData.get( 'key4' )

--> "data"

localData.set( 'key5', [1,2,3,4,9] )

localData.get( 'key5' )

--> [1, 2, 3, 4, 9]

localData.set( 'key6', new Date() )

localData.get( 'key6' )

--> Mon May 01 2017 14:39:11 GMT-0400 (Eastern Daylight Time)

localData.set( 'key7', {'a': [1,2,3] } )

localData.get( 'key7' )

--> Object {a: Array(3)}

 
get the "size" of a key's value (codepoints)

localData.size( 'key4' )

--> 4
total codepoints in value (not length, not graphemes)

 
results when querying a non-existing key

localData.forceget( 'non-existing key' )

--> null
same as localStorage.getItem( 'non-existing key' )

localData.get( 'non-existing key' )

--> undefined
the key is undefined because it does not exist, it is NOT null

localData.chopget( 'non-existing key' )

--> undefined

localData.safeget( 'non-existing key' )

--> undefined

 
read then delete a key

x = localData.chopget( 'key7' )

--> Object {a: Array(3)}

localData.get( 'key7' )

--> undefined

 
don't overwrite an existing key

localData.softset( 'key4', 'new data' )

localData.get( 'key4' )

--> "data"

 
set/get key, bypassing any data type embedding, but still observing key prefixes

localData.forceset( 'api', 13579 )

all values are stored as strings, in this case under the key passphrase.life.api

localData.forceget( 'api' )

--> "13579"

localData.forceget( 'key6' )

--> ""2017-05-01T18:39:11.443Z""

 
find duplicate key values

localData.set( 'key8', 'data' )

now key4 and key8 have the same values

localData.countdupes()

--> 1

localData.showdupes()

--> ["data"]
this key value occurs twice minimum

 
// handling duplicates; localData vs localStorage API

localData.forceset( 'dupekey1', 1234 )

will be stored as a string

localData.forceset( 'dupekey2', '1234' )

will be stored as a string

 
// look for duplicates (among localStorage keys)

localData.showdupes()

--> [1234, "data"]

 
// remove a key

localData.remove( 'dupekey1' )

prep

localData.remove( 'dupekey2' )

prep

localData.remove( 'key8' )

prep

 

localData.set( 'dupekey3', 1234 )

stored as string, but recognized as integer

localData.set( 'dupekey4', '1234' )

stored and recognized as string

 
// look for duplicates (among localData types)

localData.showdupes()

--> []
since data types are respected, no dupes were found

 

localData.set( 'dupekey1', 1234 )

prep

localData.set( 'dupekey2', '1234' )

prep

localData.set( 'key8', 'data' )

prep

 

localData.countdupes()

--> 3

localData.listdupes()

--> Object {dupecount: 3, dupes: Object}

localData.listdupes().dupecount

--> 3

localData.listdupes().dupes

--> Object {0: Object, 1: Object, 2: Object}

localData.listdupes().dupes[0]

--> Object {value: 1234, keys: Array(2)}

localData.listdupes().dupes[0].value

--> 1234

localData.listdupes().dupes[0].keys

--> ["dupekey1", "dupekey3"]

 
check if key exists

localData.haskey( 'dupekey3' )

--> true

 
check if value exists

localData.hasval( 1234 )

--> true
checks value AND data type

 

localData.set( 'testkey', 89.221 )

prep

localData.hasval( '89.221' )

--> false
the float (number) type does not match the string type

 

localData.forceset( 'LSkey1', 98765 )

set key value using localStorage API (handled as string)

localData.forcehasval( 98765 )

--> true

localData.forcehasval( '98765' )

--> true
localStorage API does not discern between data types

 

localData.hasval( 98765 )

--> true
localData attempts to coerce any value not explicity set by it

localData.hasval( '98765' )

--> false
localData will first coerce a value to a number, if possible

 
show key's value type

localData.showtype( 'dupekey3' )

--> "integer"

localData.showtype( 'dupekey4' )

--> "string"

localData.showtype( 'key1' )

--> "float"

localData.showtype( 'key3' )

--> "boolean"

localData.showtype( 'key5' )

--> "array"

localData.showtype( 'key6' )

--> "date"

localData.set( 'key7', {'local' : ['d', 'a', 't', 'a']} )

prep

localData.showtype( 'key7' )

--> "object"

 
boolean check the data type of a key's value

localData.isarray( 'key5' )

--> true

localData.isfloat( 'testkey' )

--> true

localData.isnumber( 'testkey' )

--> true

 
query by key value, not key name (returns first found)

localData.showkey( 1234 )

--> "dupekey1"

localData.showkey( '1234' )

--> "dupekey2"

 
// returns all found

localData.showkeys( 1234 )

--> ["dupekey1", "dupekey3"]

 
using the global scramble key for obfuscation

localData.getscramblekey()

--> 123456789
default global scramble key (integer)

localData.safeset( 'ss1', '007' )

--> (stored scrambled)

localData.safeget( 'ss1' )

--> "007"

 

localData.setscramblekey( new Date() )

// set global scramble key to the current date, as date object

localData.getscramblekey()

--> Mon May 01 2017 22:28:11 GMT-0400 (Eastern Daylight Time)

 

localData.safeget( 'ss1' )

--> (garbled data)
different global scramble key used for retrieval

 
// using an individual scramble key for obfuscation

localData.safeset( 'ss2', 'test', {'scramble': ['key']} )

--> (stored scrambled)
scramble keys can be any value and of any data type

localData.safeget( 'ss2', {'scramble': ['key']} )

--> "test"

 

localData.safeget( 'ss1', 123456789 )

-> "007"

 
safeget will not retrieve an unscrambled key

localData.safeget( 'key4' )

--> (garbled data)

 
renaming keys // non-scambled keys can safely be renamed

localData.rename( 'key4', 'key4-renamed' )

key4 no longer exists

localData.get( 'key4' )

--> undefined

localData.get( 'key4-renamed' )

--> "data"

 
// scrambled keys cannot be renamed: the key name and the value together produce the obfuscation

localData.rename( 'ss1', 'ss1-renamed' )

key ss1 no longer exists

localData.safeget( 'ss1' )

--> undefined

localData.safeget( 'ss1-renamed', 123456789 )

--> (garbled data)
this was the correct scramble key for the 'ss1' key, but not for the 'ss1-renamed' key

 

localData.rename( 'ss1-renamed', 'ss1' )
key ss1-renamed no longer exists

localData.safeget( 'ss1-renamed' )

--> undefined

localData.safeget( 'ss1', 123456789 )

--> "007"

 
how localDataStorage reacts to values set via the localStorage API

localData.forceset( 'lsAPIkey', 77.042 )

always stored as a string by the native API

localData.forceget( 'lsAPIkey' )

--> "77.042"

localData.get( 'lsAPIkey' )

--> 77.042
localData will coerce value to number when possible

localData.showtype( 'lsAPIkey' )

--> "presumed number"
('presumed' because value was coerced, not set)

 
there are several ways to track memory usage

// show memory required to store key value

localData.showtype( 'dupekey4' )

--> "string";

localData.get( 'dupekey4' )

--> "1234"

localData.size( 'dupekey4' )

--> 4

localData.valbytes( 'dupekey4' )

--> "8.00 bytes"
localStorage uses 16 bits to store 1 byte (only the data is counted)

localData.valbytesall( 'dupekey4' )

--> "12.00 bytes"
now we include the 2-byte embedded marker (total data)

 
// show memory required to store key name

localData.keybytes( 'dupekey4' )

--> "48.00 bytes"
the prefix ('passphrase.life' + '.') is 32 bytes, plus key name is 16 bytes more ('dupekey4' ), yielding 48 bytes

 
// show memory used by the key-value pair

// key name + raw value

localData.bytes( 'dupekey4' )

--> "56.00 bytes"
8 bytes for raw value and 48 bytes for name, i.e. valbytes + keybytes

 
// key name + total value (include value marker byte)

localData.bytesall( 'dupekey4' )

--> "60.00 bytes"
now includes the embedded data type marker (it's 2 bytes, stored as 4)

 
view memory usage of compressed key values

localData.set( 'crunchedkey', 'this is some test data' )

only strings can be compressed; other data types will ignore compression

localData.size( 'crunchedkey' )

--> 22

localData.valbytes( 'crunchedkey' )

--> "44.00 bytes"
memory used to store raw string of 22 graphemes (each is 7-bit ASCII)

localData.valbytesall( 'crunchedkey' )

--> "34.00 bytes"
total memory required to store compressed string + embedded data type marker

 
unicode-safe data storage

localData.set( 'unicodeKey1', '😀' )

storing an emoji; 1 grapheme (1 codepoint in 4 bytes)

localData.get( 'unicodeKey1' )

--> "😀"

localData.size( 'unicodeKey1' )

--> 1; one codepoint

localData.valbytes( 'unicodeKey1' )

--> "8.00 bytes"

localData.valbytesall( 'unicodeKey1' )

--> "12.00 bytes"

 

localData.set( 'unicodeKey2', '🕔🔚🔈🔔♅' )

storing 5 graphemes (5 codepoints in 19 bytes)

localData.get( 'unicodeKey2' )

--> "🕔🔚🔈🔔♅"

localData.size( 'unicodeKey2' )

--> 5

localData.valbytes( 'unicodeKey2' )

--> "38.00 bytes"

localData.valbytesall( 'unicodeKey2' )

--> "42.00 bytes"

 
// using emojis for key name, key value and individual scramble key

localData.safeset( '👊🌐🔷', '💕🚻', '🔙' )

localData.safeget( '👊🌐🔷', '🔙' )

--> "💕🚻"

 
// using emojis in the global scramble key

localData.setscramblekey( '🎵🎶🔶🔻' )

localData.safeset( 'Ron Wyden', '.@NSAGov 💻📱📡📞🔎👂👀🔚 #EndThisDragnet' )

localData.safeget( 'Ron Wyden' )

--> ".@NSAGov 💻📱📡📞🔎👂👀🔚 #EndThisDragnet"

 
get tally of keys

localData.keys()

--> 24

 
delete all prefixed keys in the domain (unprefixed localStorage keys are not affected)

localStorage.setItem( 'API-key', 'test data' )

create a key in the same domain completely outside our instance of localDataStorage

localData.clear()

--> "24 keys removed"

localStorage.getItem( 'API-key' )

--> "test data"
any unprefixed localStorage keys are untouched

localData.safeget( 'Ron Wyden' )

--> undefined
all localData keys have been removed

Tested:

Google Chrome on Win 8.1 (x64)
 

Version notes:

• 1.2.0 - 19 JUN 2017

NEW: Check if localStorage is available and, if not, gracefully fails when called. This means that all methods will simply return false instead of nasty type errors.
 

• 1.1.0 - 17 MAY 2017

NEW: Add ability to listen to key value change events (in same window/tab).
 

• 1.0.0 - 15 MAY 2017

Initial release.
 

License (BSD)

Copyright (c) 2017, William McMeans
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.