As of August 2020 the site you are on (wiki.newae.com) is deprecated, and content is now at rtfm.newae.com.

Difference between revisions of "File Formats"

From ChipWhisperer Wiki
Jump to: navigation, search
(→‎Native File Format: Needs scripts, but initial documentation done)
(→‎Example Scripts: Add multi-trace example, untested)
Line 118: Line 118:
  
 
== Native File Format to MATLAB Workspace ==
 
== Native File Format to MATLAB Workspace ==
 +
The following will convert a single file into a MATLAB workspace:<syntaxhighlight lang="python">
 +
import scipy.io
 +
import numpy as np
 +
 +
def native_to_matlab(prefix):
 +
    tracedata = np.load(prefix + '_traces.npy')
 +
    textin = np.load(prefix + '_textin.npy')
 +
    textout = np.load(prefix + '_textout.npy')
 +
    key = np.load(prefix + '_keylist.npy')
 +
 +
    scipy.io.savemat('fileout.mat', {
 +
    "traces":tracedata,
 +
    "textin":textin,
 +
    "textout":textout,
 +
    "key":key
 +
    })
 +
 +
prefix = r'2016.02.24-19.07.03'
 +
#Can point to full directory like:
 +
#prefix = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\2016.02.24-19.07.03'
 +
 +
native_to_matlab(prefix)
 +
</syntaxhighlight>This file can be saved in a location where you want the output MATLAB file. You can specify the full directory link if you are not saving the MATLAB file to the same location as the .npy file. This will operate on a single trace-file only, not on an entire set. If you wish to operate on an entire set, you can load all such files and save the output once like this:<syntaxhighlight lang="python">
 +
import scipy.io
 +
import numpy as np
 +
 +
class native_to_matlab(object):
 +
   
 +
    def __init__(self):
 +
        self.tracedata = None
 +
        self.textin = None
 +
        self.textout = None
 +
        self.key = None
 +
       
 +
    def add_trace(self, prefix):
 +
        _tracedata = np.load(prefix + '_traces.npy')
 +
        _textin = np.load(prefix + '_textin.npy')
 +
        _textout = np.load(prefix + '_textout.npy')
 +
        _key = np.load(prefix + '_keylist.npy')
 +
       
 +
        if self.tracedata is None:
 +
            self.tracedata = _tracedata
 +
            self.textin = _textin
 +
            self.textout = _textout
 +
            self.key = _key
 +
           
 +
        else:
 +
            self.tracedata.append(_tracedata)
 +
            self.textin.append(_textin)
 +
            self.textout.append(_textout)
 +
            self.key.append(_key)
 +
   
 +
    def write_matlab(self, fname):
 +
        scipy.io.savemat(fname, {
 +
        "traces":self.tracedata,
 +
        "textin":self.textin,
 +
        "textout":self.textout,
 +
        "key":self.key
 +
        })
 +
 +
n2m = native_to_matlab()
 +
   
 +
prefix_list = [r'2016.02.24-19.07.03'
 +
#Can point to full directory like:
 +
root_dir = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\'
 +
 +
for prefix in prefix_list:
 +
    n2m.add_trace(root_dir + prefix)
 +
   
 +
n2m.write_matlab('fileout.mat')
 +
</syntaxhighlight>
  
 
== Native File Format to Text Files ==
 
== Native File Format to Text Files ==

Revision as of 09:54, 18 May 2016

File Formats (Projects and Traces)

This page explains some of the file formats used in the ChipWhisperer Software.

Project Files

Project files are stored as a simple text-file. The most basic file will include information about stored traces, along with some basic project file information.

As an example, the following shows a simple project file named "test_cw305_artix100_10k_4seg.cwp":
[Trace Management]
tracefile0 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.21.25_.cfg
enabled0 = True
tracefile1 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.26.48_.cfg
enabled1 = True
tracefile2 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.32.03_.cfg
enabled2 = True
tracefile3 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.37.27_.cfg
enabled3 = True
[ChipWhisperer]
[[General Settings]]
Project Name = Untitled
Program Name = ChipWhisperer
Project File Version = 1.00
Project Author = Unknown
Program Version = V3.0
The most important files are the ones contained in the [Trace Management] section. These contain links to the trace data stored in the project. The specific link is to a configuration file, which is described below.

Normally, in the same folder that the .cwp file is located will be a folder with the same suffix. For example if you have a file called supersecret_aes_test.cwp, there will be a directory called supersecret_aes_test. This directory will contain the trace data along with additional files generated during analysis. Note it's not required that all files are located in this directory, as you can reference another location.

Trace Files

Trace files in the ChipWhisperer software are defined through a configuration file, with the suffix .cfg. Any trace added to the ChipWhisperer project will have a configuration file - this file does not store data, but tells the software where it is stored and what format it is stored in.

While ChipWhisperer has a "native" file format, you can also interface to existing files. We'll first look at the configuration file before exploring the various trace file formats supported.

The most basic trace file configuration has this format, for example this file is config_2016.05.15-18.21.25_.cfg that was referenced earlier:
[Trace Config]
numTraces = 2500
format = native
numPoints = 240
prefix = 2016.05.15-18.21.25_
date = 2016-05-15 18:21:25
scopeSampleRate = 0
notes = ""
scopeName = ChipWhisperer/OpenADC
scopeXUnits = 0
targetSW = unknown
targetHW = ChipWhisperer CW305 (Artix-7)
scopeYUnits = 0
The most important lines are the first three (note - order is NOT important, only the name is used as an index), shown here:
Option Name Option Description Format
numTraces Number of traces contained in this reference. Int
format The format used to store the traces. String
numPoints Number of samples (points) within each trace. Int

The remaining options are used to populate the "trace manager" window.

Let's now investigate specific options for the "native" file format, along with format information.

Native File Format

Specific options for the native file format are:

Option Name Option Description Format
prefix Prefix for the files (trace, textin, textout, etc) String

The prefix declares specific files. For example the prefix in the above example is 2016.05.15-18.21.25_, which means data files will start with 2016.05.15-18.21.25_.

The suffix for data files comes from the type of file, given in this table:

File Suffix File Description File Type
keylist.npy List of encryption key used for each trace. If the same key is used for each trace, will be the same key repeated N times.
knownkey.npy If same key used for all traces, contains the encryption key.
textin.npy Input data for each trace (i.e., plaintext if encryption).
textout.npy Output data for each trace (i.e., ciphertext if encryption).
traces.npy Power measurement for each trace.

Each of these is a NumPy array, where the first element is trace n=0, the second is trace n=1, etc. You can load this file using the NumPy load command.

Example Scripts

Load Native File Format, modify a value, Save File(s)

Native File Format to MATLAB Workspace

The following will convert a single file into a MATLAB workspace:
import scipy.io
import numpy as np

def native_to_matlab(prefix):
    tracedata = np.load(prefix + '_traces.npy')
    textin = np.load(prefix + '_textin.npy')
    textout = np.load(prefix + '_textout.npy')
    key = np.load(prefix + '_keylist.npy')

    scipy.io.savemat('fileout.mat', {
     "traces":tracedata,
     "textin":textin,
     "textout":textout,
     "key":key
    })

prefix = r'2016.02.24-19.07.03'
#Can point to full directory like:
#prefix = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\2016.02.24-19.07.03'

native_to_matlab(prefix)
This file can be saved in a location where you want the output MATLAB file. You can specify the full directory link if you are not saving the MATLAB file to the same location as the .npy file. This will operate on a single trace-file only, not on an entire set. If you wish to operate on an entire set, you can load all such files and save the output once like this:
import scipy.io
import numpy as np

class native_to_matlab(object):
    
    def __init__(self):
        self.tracedata = None
        self.textin = None
        self.textout = None
        self.key = None
        
    def add_trace(self, prefix):
        _tracedata = np.load(prefix + '_traces.npy')
        _textin = np.load(prefix + '_textin.npy')
        _textout = np.load(prefix + '_textout.npy')
        _key = np.load(prefix + '_keylist.npy')
        
        if self.tracedata is None:
            self.tracedata = _tracedata
            self.textin = _textin
            self.textout = _textout
            self.key = _key
            
        else:
            self.tracedata.append(_tracedata)
            self.textin.append(_textin)
            self.textout.append(_textout)
            self.key.append(_key)
    
    def write_matlab(self, fname):
        scipy.io.savemat(fname, {
         "traces":self.tracedata,
         "textin":self.textin,
         "textout":self.textout,
         "key":self.key
        })

n2m = native_to_matlab()
    
prefix_list = [r'2016.02.24-19.07.03'
#Can point to full directory like:
root_dir = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\'

for prefix in prefix_list:
    n2m.add_trace(root_dir + prefix)
    
n2m.write_matlab('fileout.mat')

Native File Format to Text Files

Retrieved from "http://wiki.newae.com/index.php?title=File_Formats&oldid=517"