Module laplace.utils.subnetmask

Classes

class SubnetMask (model)

Baseclass for all subnetwork masks in this library (for subnetwork Laplace).

Parameters

model : torch.nn.Module

 

Subclasses

• ModuleNameSubnetMask
• ParamNameSubnetMask
• laplace.utils.subnetmask.ScoreBasedSubnetMask

Instance variables

var indices

var n_params_subnet

Methods

def convert_subnet_mask_to_indices(self, subnet_mask)

Converts a subnetwork mask into subnetwork indices.

Parameters

subnet_mask : torch.Tensor

a binary vector of size (n_params) where 1s locate the subnetwork parameters within the vectorized model parameters (i.e. torch.nn.utils.parameters_to_vector(model.parameters()))

Returns

subnet_mask_indices : torch.LongTensor

a vector of indices of the vectorized model parameters (i.e. torch.nn.utils.parameters_to_vector(model.parameters())) that define the subnetwork

def select(self, train_loader=None)

Select the subnetwork mask.

Parameters

train_loader : torch.data.utils.DataLoader, default=None

each iterate is a training batch (X, y); train_loader.dataset needs to be set to access $N$, size of the data set

Returns

subnet_mask_indices : torch.LongTensor

a vector of indices of the vectorized model parameters (i.e. torch.nn.utils.parameters_to_vector(model.parameters())) that define the subnetwork

def get_subnet_mask(self, train_loader)

Get the subnetwork mask.

Parameters

train_loader : torch.data.utils.DataLoader

each iterate is a training batch (X, y); train_loader.dataset needs to be set to access $N$, size of the data set

Returns

subnet_mask : torch.Tensor

a binary vector of size (n_params) where 1s locate the subnetwork parameters within the vectorized model parameters (i.e. torch.nn.utils.parameters_to_vector(model.parameters()))

class RandomSubnetMask (model, n_params_subnet)

Subnetwork mask of parameters sampled uniformly at random.

Ancestors

• laplace.utils.subnetmask.ScoreBasedSubnetMask
• SubnetMask

Methods

def compute_param_scores(self, train_loader)

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • get_subnet_mask
  • select

class LargestMagnitudeSubnetMask (model, n_params_subnet)

Subnetwork mask identifying the parameters with the largest magnitude.

Ancestors

• laplace.utils.subnetmask.ScoreBasedSubnetMask
• SubnetMask

Methods

def compute_param_scores(self, train_loader)

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • get_subnet_mask
  • select

class LargestVarianceDiagLaplaceSubnetMask (model, n_params_subnet, diag_laplace_model)

Subnetwork mask identifying the parameters with the largest marginal variances (estimated using a diagonal Laplace approximation over all model parameters).

Parameters

model : torch.nn.Module

 

n_params_subnet : int

number of parameters in the subnetwork (i.e. number of top-scoring parameters to select)

diag_laplace_model : DiagLaplace

diagonal Laplace model to use for variance estimation

Ancestors

• laplace.utils.subnetmask.ScoreBasedSubnetMask
• SubnetMask

Methods

def compute_param_scores(self, train_loader)

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • get_subnet_mask
  • select

class LargestVarianceSWAGSubnetMask (model, n_params_subnet, likelihood='classification', swag_n_snapshots=40, swag_snapshot_freq=1, swag_lr=0.01)

Subnetwork mask identifying the parameters with the largest marginal variances (estimated using diagonal SWAG over all model parameters).

Parameters

model : torch.nn.Module

 

n_params_subnet : int

number of parameters in the subnetwork (i.e. number of top-scoring parameters to select)

likelihood : str

'classification' or 'regression'

swag_n_snapshots : int

number of model snapshots to collect for SWAG

swag_snapshot_freq : int

SWAG snapshot collection frequency (in epochs)

swag_lr : float

learning rate for SWAG snapshot collection

Ancestors

• laplace.utils.subnetmask.ScoreBasedSubnetMask
• SubnetMask

Methods

def compute_param_scores(self, train_loader)

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • get_subnet_mask
  • select

class ParamNameSubnetMask (model, parameter_names)

Subnetwork mask corresponding to the specified parameters of the neural network.

Parameters

model : torch.nn.Module

 

parameter_names : List[str]

list of names of the parameters (as in model.named_parameters()) that define the subnetwork

Ancestors

• SubnetMask

Methods

def get_subnet_mask(self, train_loader)

Get the subnetwork mask identifying the specified parameters.

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • select

class ModuleNameSubnetMask (model, module_names)

Subnetwork mask corresponding to the specified modules of the neural network.

Parameters

model : torch.nn.Module

 

parameter_names : List[str]

list of names of the modules (as in model.named_modules()) that define the subnetwork; the modules cannot have children, i.e. need to be leaf modules

Ancestors

• SubnetMask

Subclasses

• LastLayerSubnetMask

Methods

def get_subnet_mask(self, train_loader)

Get the subnetwork mask identifying the specified modules.

Inherited members

• SubnetMask:
  • convert_subnet_mask_to_indices
  • select

class LastLayerSubnetMask (model, last_layer_name=None)

Subnetwork mask corresponding to the last layer of the neural network.

Parameters

model : torch.nn.Module

 

last_layer_name : str, default=None

name of the model's last layer, if None it will be determined automatically

Ancestors

• ModuleNameSubnetMask
• SubnetMask

Methods

def get_subnet_mask(self, train_loader)

Get the subnetwork mask identifying the last layer.

Inherited members

• ModuleNameSubnetMask:
  • convert_subnet_mask_to_indices
  • select