Reference: pipeline elements

Working on the client side

Generating fake client data

snap.elements.client.fake.realtime(shift: float = 0)

Output each timestamp from the list at their appropriate time

Input

list of unix timestamps

Output

yield each timestamp individually at their time.

Parameters

shift – time shift in seconds

snap.elements.client.fake.sample_ts(B, S=0, tSN=0, tChunk=10)

Data source, generating random event timestamps following the given rates, with signal injection.

Parameters

  • B (rate) – Background rate vs. time

  • S (rate) – Signal rate vs. time from signal start.

  • tSN (float) – Delay of the signal to start, after the generator started work.

Yields

event timestamp in seconds

Calculating significance time series from event timestamps

class snap.elements.client.CountAna(B, time_window, *, dt=0.1, tChunk_min=1, delay=0)

Processing step: counting analysis to calculate significance

Parameters

  • B (rate) – Background rate

  • time_window (tuple (float, float)) – A relative time window to count the interactions, for example [-5,5]. Only interactions within the time window affect current significance.

Keyword Arguments

  • dt (float) – time step, seconds

  • tChunk_min (float) – minimal time duration of the produced chunk of data

  • delay (float) –

    processing delay in seconds. This means that maximum assumed supernova time will be:

    now()-delay-ana.time_window[1]

Input

data (list of float): list of events’ timestamps

Output
snap.datablock.DataBlock

with the SN observation significance

class snap.elements.client.ShapeAna(B, S, time_window='auto', *, dt=0.1, tChunk_min=1, delay=0)

Processing step: shape analysis to calculate significance

Parameters

  • B (rate) – Background rate

  • S (rate) – Signal rate

  • time_window (tuple (float, float) or "auto") – A relative time window to count the interactions, for example [-5,5]. Only interactions within the time window affect current significance. If “auto”, try to get the range from signal shape.

Keyword Arguments

  • dt (float) – time step, seconds

  • tChunk_min (float) – minimal time duration of the produced chunk of data

  • delay (float) –

    processing delay in seconds. This means that maximum assumed supernova time will be:

    now()-delay-ana.time_window[1]

Input

data (list of float): list of events’ timestamps

Output
snap.datablock.DataBlock

with the SN observation significance

Setting client ID

class snap.elements.client.setId(id: str)

Processing step, setting given id to all the passing data objects.

Parameters

id – client id to be set.

Input

data (object)

Output

data with set id (data.id = id)

Significance combination

Buffering

class snap.elements.combine.Buffer(delay: float = 1, timeout: float = 6000)

Buffer which stores the incoming DataBlocks with multiple client ids, and provides the time slice of all the data in a recently updated region.

Parameters

  • delay (float) – Time to wait before processing a new region (seconds)

  • timeout (float) – Defines how long will the data stay in the buffer (on :put:)

Input

DataBlock with significance time series for one client. Datablock should have id properly set. When the data with time span [t0,t1] arrives, this region is set as “updated”, and marked for the output later.

Output

DataBlock with all the available clients’ data in an updated region [t0,t1]. After the data in this region has been output, it’s no longer marked “updated”, until new data arrives in that time region.

Combination methods

class snap.elements.combine.Fisher(data: snap.datablock.DataBlock)

Fisher combination method, based on calculating test statistics \(X^2_{2N} = -2\sum_i \log (p_i)\)

which follows chi2 distribution, where N is number of clients combined.

Input

DataBlock with significance for one or many clients

Output

DataBlock with combined significance, for the same time stamps as input.

class snap.elements.combine.Stouffer(weights: dict)

Stouffer combination method: combined significance is linear combination of all input significances: \(z_c = \sum_i(z_i \cdot w_i)\)

Parameters

weights – dict (srt, float) Mapping det_id: w_i, providing the coefficients for linear combination.

Note

Weights don’t need to be normalized: the normalization is calculated depending on which detector ids are present in a given datablock id

Input

DataBlock with significance for one or many clients

Output

DataBlock with combined significance, for the same time stamps as input.

Processing and analyzing incoming data

class snap.elements.process.SmartAlert(threshold: float = 5, timeout: float = 600)

A precessing step for detecting the parts (clusters) of the time series above threshold, bookkeeping of these parts and producing ‘NEW/UPD/DEL’ commands for these clusters.

Parameters

  • thr (float) – significance threshold

  • timeout (float) – time in seconds for which the data is kept in buffer

Input

data (DataBlock) with the time series (zs vs ts)

Output

DataBlock containing cluster of values with zs above threshold. Will not output until such cluster is found.

Example

If the incoming data was:

ts=[0,1,2,3,4,5,6,7,8,9,10]
zs=[0,1,2,2,1,0,1,2,3,2,1 ]

Then the found clusters for thr=1 should be:

1) ts=[2,3],  zs=[2,2]
2) ts=[7,8,9],zs=[2,3,4]

And they will be output in separate iterations.

update_clusters(clusters)

Update the current clusters with the new ones. This means:

UPDATE old cluster, if it collides with a new cluster DELETE old cluster, if it doesn’t collide to new clusters CREATE new cluster if it didn’t collide with any old ones

Parameters

clusters (list) – List of new clusters found

Returns

‘DEL’: list of clusters to delete ‘UPD’: list of clusters to update ‘NEW’: list of clusters to create

Return type

dict