Request efficient encoding for plotly offline

[cboulay]

I know @jonmmease is planning to do this anyway, so this issue isn't really necessary. However, I said that I would create it and I'd like to humbly ask to please keep multi-dimensional arrays in the back of your mind as you work on this. Heatmaps and contour plots are my biggest offenders.

[jonmmease]

Thanks for creating the issue @cboulay ,

In the meantime, let's do an experiment. I'd like to see how much the base64 encoding would help in your situation.

Take one of the large multi-dimensional numpy arrays that you're dealing with, lets call it a. First convert it to JSON as a list and measure the time and size (in ipython or jupyter):

import json
from plotly.utils import PlotlyJSONEncoder
encode_timing1 = %timeit -o json.dumps(a, cls=PlotlyJSONEncoder)
res1 = json.dumps(a, cls=PlotlyJSONEncoder)
decode_timing1 = %timeit -o json.loads(res1)

Then convert to a base64 encoded string:

import base64
encode_timing2 = %timeit -o base64.b64encode(memoryview(a)).decode()
res2 = base64.b64encode(memoryview(a)).decode()
decode_timing2 = %timeit -o np.frombuffer(base64.b64decode(res2.encode('utf-8')), dtype=a.dtype)
len(res2)

Then compare

print('Size list: %d chars' % len(res1))
print('Size base64: %d chars' % len(res2))
print('Size ratio: %.4f' % (len(res2)/len(res1)))

print('Encode time list: %.6f seconds' % encode_timing1.average)
print('Encode time base64: %.6f seconds' % encode_timing2.average)
print('Encode time ratio: %.6f' % (encode_timing2.average / encode_timing1.average))

print('Decode time list: %.6f seconds' % decode_timing1.average)
print('Decode time base64: %.6f seconds' % decode_timing2.average)
print('Decode time ratio: %.6f' % (decode_timing2.average / decode_timing1.average))

[gvwilson]

Hi - we are currently trying to tidy up Plotly's public repositories to help us focus our efforts on things that will help users most. Since this issue has been sitting for several years, so I'm going to close it. If it's still a concern, we'd be grateful if you could open a new issue (with a short reproducible example if appropriate) so that we can add it to our backlog. Thanks for your help - @gvwilson