File Formats (Projects and Traces)

This page explains some of the file formats used in the ChipWhisperer Software.

Project Files

Project files are stored as a simple text-file. The most basic file will include information about stored traces, along with some basic project file information.

As an example, the following shows a simple project file named "test_cw305_artix100_10k_4seg.cwp":

[Trace Management]
tracefile0 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.21.25_.cfg
enabled0 = True
tracefile1 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.26.48_.cfg
enabled1 = True
tracefile2 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.32.03_.cfg
enabled2 = True
tracefile3 = test_cw305_artix100_10k_4seg_data\traces\config_2016.05.15-18.37.27_.cfg
enabled3 = True
[ChipWhisperer]
[[General Settings]]
Project Name = Untitled
Program Name = ChipWhisperer
Project File Version = 1.00
Project Author = Unknown
Program Version = V3.0

The most important files are the ones contained in the [Trace Management] section. These contain links to the trace data stored in the project. The specific link is to a configuration file, which is described below.

Normally, in the same folder that the .cwp file is located will be a folder with the same suffix. For example if you have a file called supersecret_aes_test.cwp, there will be a directory called supersecret_aes_test. This directory will contain the trace data along with additional files generated during analysis. Note it's not required that all files are located in this directory, as you can reference another location.

If you perform capture before saving a project file, files will be written to a temporary directory. In this case your project file will reference this temporary directory. Files may not be automatically moved out of this temporary directory when you save a project, but you can always use the "Consolidate" option to force all data to be moved into the project file directory.

Trace Files

Trace files in the ChipWhisperer software are defined through a configuration file, with the suffix .cfg. Any trace added to the ChipWhisperer project will have a configuration file - this file does not store data, but tells the software where it is stored and what format it is stored in.

While ChipWhisperer has a "native" file format, you can also interface to existing files. We'll first look at the configuration file before exploring the various trace file formats supported.

The most basic trace file configuration has this format, for example this file is config_2016.05.15-18.21.25_.cfg that was referenced earlier:

[Trace Config]
numTraces = 2500
format = native
numPoints = 240
prefix = 2016.05.15-18.21.25_
date = 2016-05-15 18:21:25
scopeSampleRate = 0
notes = ""
scopeName = ChipWhisperer/OpenADC
scopeXUnits = 0
targetSW = unknown
targetHW = ChipWhisperer CW305 (Artix-7)
scopeYUnits = 0

The most important lines are the first three (note - order is NOT important, only the name is used as an index), shown here:

Option Name	Option Description	Format
numTraces	Number of traces contained in this reference.	Int
format	The format used to store the traces.	String
numPoints	Number of samples (points) within each trace.	Int

The remaining options are used to populate the "trace manager" window.

Let's now investigate specific options for the "native" file format, along with format information.

Native File Format

Specific options for the native file format are:

Option Name	Option Description	Format
prefix	Prefix for the files (trace, textin, textout, etc)	String

The prefix declares specific files. For example the prefix in the above example is 2016.05.15-18.21.25_, which means data files will start with 2016.05.15-18.21.25_.

The suffix for data files comes from the type of file, given in this table:

File Suffix	File Description	File Type
keylist.npy	List of encryption key used for each trace. If the same key is used for each trace, will be the same key repeated N times.
knownkey.npy	If same key used for all traces, contains the encryption key.
textin.npy	Input data for each trace (i.e., plaintext if encryption).
textout.npy	Output data for each trace (i.e., ciphertext if encryption).
traces.npy	Power measurement for each trace.

Each of these is a NumPy array, where the first element is trace n=0, the second is trace n=1, etc. You can load this file using the NumPy load command.

Example Scripts

Load Native File Format, modify a value, Save File(s)

Native File Format to MATLAB Workspace

The following will convert a single file into a MATLAB workspace:

import scipy.io
import numpy as np

def native_to_matlab(prefix):
    tracedata = np.load(prefix + '_traces.npy')
    textin = np.load(prefix + '_textin.npy')
    textout = np.load(prefix + '_textout.npy')
    key = np.load(prefix + '_keylist.npy')

    scipy.io.savemat('fileout.mat', {
     "traces":tracedata,
     "textin":textin,
     "textout":textout,
     "key":key
    })

prefix = r'2016.02.24-19.07.03'
#Can point to full directory like:
#prefix = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\2016.02.24-19.07.03'

native_to_matlab(prefix)

This file can be saved in a location where you want the output MATLAB file. You can specify the full directory link if you are not saving the MATLAB file to the same location as the .npy file. This will operate on a single trace-file only, not on an entire set. If you wish to operate on an entire set, you can load all such files and save the output once like this:

import scipy.io
import numpy as np

class native_to_matlab(object):
    
    def __init__(self):
        self.tracedata = None
        self.textin = None
        self.textout = None
        self.key = None
        
    def add_trace(self, prefix):
        _tracedata = np.load(prefix + '_traces.npy')
        _textin = np.load(prefix + '_textin.npy')
        _textout = np.load(prefix + '_textout.npy')
        _key = np.load(prefix + '_keylist.npy')
        
        if self.tracedata is None:
            self.tracedata = _tracedata
            self.textin = _textin
            self.textout = _textout
            self.key = _key
            
        else:
            self.tracedata.append(_tracedata)
            self.textin.append(_textin)
            self.textout.append(_textout)
            self.key.append(_key)
    
    def write_matlab(self, fname):
        scipy.io.savemat(fname, {
         "traces":self.tracedata,
         "textin":self.textin,
         "textout":self.textout,
         "key":self.key
        })

n2m = native_to_matlab()
    
prefix_list = [r'2016.02.24-19.07.03'
#Can point to full directory like:
root_dir = r'C:\Temp\demo\traces_capdir48\knownrand_fixed\knownrand_fixed_P48_data\traces\'

for prefix in prefix_list:
    n2m.add_trace(root_dir + prefix)
    
n2m.write_matlab('fileout.mat')

File Formats