SOURCE CODE DOC¶

Entry Point¶

Package import¶

PSPMAN: PSeudo Package Manager

pspman.CONFIG = <pspman.config.MetaConfig object>¶: Meta data information about C_DIR(s), configuration directory, etc

pspman.ENV = Clone Directory: /home/docs/.pspman/src Prefix: /home/docs/.pspman Called sub-function: None Clean-Resets Requested: 0 Deletions Requested: 0 Additions requested: 0 Optional Flags: Risk Root: False Only Pull: False Don't Update: False Verbose Debugging: False ¶: Standard installation context

pspman.print(*args, mark=None, **kwargs)¶

Customized psprint function

Return type: None

Call function¶

Object Framework¶

classes¶

object classes

class pspman.classes.GitProjEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]¶

Encode Class object’s __dict__

default(o)[source]¶

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

Return type: dict

class pspman.classes.GitProject(**kwargs)[source]¶

Git project object.

Parameters

**kwargs –

args corresponding to Attributes: hard set
data: Dict[str, Any]: load data
rest are ignored

url¶: url for git remote

name¶: name of project folder

tag¶: action tagged to project

branch¶: git branch to be cloned

sh_env¶: environ modifications before installation

inst_argv¶: arguments suffixed to optional args before positional args

pull¶: only pull this project, don’t run install scripts

last_updated¶: last updated on datetime

mark_update_time()[source]¶: Mark that the project was updated

merge(data)[source]¶

Update values that are None type using data. Doesn’t change set values

Parameters: data (Dict[str, object]) – source for update
Return type: None

update_name()[source]¶

Update project name. Call this method after updating url

Return type: str
Returns: The updated name

class pspman.classes.InstallEnv(config, **kwargs)[source]¶

Installation variables

call_function¶: sub-function called {version,info,meta,unlock}

clone_dir¶: base directory to clone src

prefix¶: prefix for installation

risk¶: risk root

pull¶: only pull, don’t try to install

stale¶: do not update, only delete/install new

verbose¶: print semi-verbose output

install¶: install (clone) from remote urls

delete¶: delete projects

Parameters

config (MetaConfig) – MetaConfig to determine default values
**kwargs – hard set attributes

update(options)[source]¶

Update attributes with those from options

Parameters: options (Dict[str, object]) – hard-set config
Return type: InstallEnv

Configuration¶

Configuration

class pspman.config.GroupDB(**kwargs)[source]¶

Group database information

Parameters

**kwargs –

data: create object using data as __dict__
arg in Attributes: Hard-set attributes
rest are ignored

grp_path¶: path to Group

name¶: custom-name for group [default: leaf of path]

exists¶: whether path exists

locked¶: locked by another process?

clone_type¶

type of clones (0[default], -1, 1)

1: all projects are installable
-1: no project is installable (all are pull-only)
0: heterogenous

get_name()[source]¶

Infer name from filepath

Return type: str

merge(data)[source]¶

Update values that are None type using data. Doesn’t change set values

Parameters: data (Dict[str, object]) – source for update
Return type: None

mk_structure()[source]¶: Generate a directory structure for the GitGroup

class pspman.config.MetaConfig(**kwargs)[source]¶

Meta-Config bearing information about all GroupDBs

Parameters

kwargs –

arg in Attributes: Hard-set attributes
rest are ignored

meta_db_dirs¶: registry of all GroupDBs

config_dir¶: pspman configuration directory

data_dir¶: pspman default data directory

opt_in¶: opted installation methods

add(group)[source]¶

Add (don’t overwrite) Group

Parameters: group (Union[GroupDB, dict]) – Group (or its __dict__) to be added

load(psp_config_file)[source]¶

Load meta configuration

Parameters: psp_config_dir – yaml file containing GroupDB meta data
Return type: bool
Returns: True if file was successfully loaded

prune()[source]¶: If any group doesn’t exist anymore, remove it

remove(name)[source]¶

Remove group

Parameters: name (str) – name of group to remove

store()[source]¶: Refresh a configuration file

pspman.config.read_config(config_file=None)[source]¶

Read and maintain default configurations

Parameters: config_file (Optional[str]) – path to file containing configurations
Return type: MetaConfig
Returns: Configuration

Serial Code¶

Action Definition by user call¶

Define variables from command line and defaults

pspman.define.cli(config=None)[source]¶

Parse command line arguments

Parameters: config (Optional[MetaConfig]) – configuration to be modified by command line inputs
Return type: ArgumentParser
Returns: modified confing

pspman.define.cli_opts(config=None)[source]¶

Parse cli arguments to return its dict

Return type: Dict[str, Any]

pspman.define.lock(env, unlock=False, message=None)[source]¶

Unlock up the directory

Parameters

env (InstallEnv) – installation context
unlock (bool) – unlock existing locks?
message (Optional[str]) – message to be written in the lockfile instead of pid

Returns

Error code

pspman.define.perm_pass(env, permdir)[source]¶

Parameters: permdir (Path) – directory whose permissions are to be checked
Returns: 1 if all rwx permissions are not granted
Return type: Error code

pspman.define.prepare_env(env)[source]¶

Check permissions and create prefix and source directories

Return type: int
Returns: Error code

Project actions¶

Actions on projects, other than automated installations

pspman.serial_actions.add_projects(env, git_projects, queues, to_add_list=None)[source]¶

Add a project with given url

Parameters

env (InstallEnv) – Installation context
git_projects (Dict[str, GitProject]) – known git projects
queues (Dict[str, PSPQueue]) – initiated queues
to_add_list (Optional[List[str]]) – urls of projects to be added

Return type

None

pspman.serial_actions.del_projects(env, git_projects, queues, del_list=None)[source]¶

Delete given project

Parameters

env (InstallEnv) – Installation context
git_projects (Dict[str, GitProject]) – known git projects
queues (Dict[str, PSPQueue]) – initiated queues
del_list (Optional[List[str]]) – list of names of project directories to be removed

Return type

Dict[str, GitProject]

Returns

Updated registry of GitProjects

pspman.serial_actions.end_queues(env, queues)[source]¶

wait (blocking) for queues (threads) to end and return

Parameters

env (InstallEnv) – Installation context
queues (Dict[str, PSPQueue]) – initiated queues

Return type

bool

pspman.serial_actions.find_gits(env, git_projects=None)[source]¶

Locate git projects in the defined environment (parse) Load database (overrides parser)

Parameters

env (InstallEnv) – Installation context
git_projects (Optional[Dict[str, GitProject]]) – Already known git projects

Return type

Tuple[Dict[str, GitProject], Dict[str, GitProject]]

Returns

All project names found in the environment

pspman.serial_actions.init_queues(env)[source]¶

Initiate success queues

Parameters: env (InstallEnv) – Installation context
Return type: Dict[str, PSPQueue]

pspman.serial_actions.interrupt(queues)[source]¶

Interrupt actions as they are, kill all children

Parameters: queues (Dict[str, PSPQueue]) – intitiated queues

pspman.serial_actions.load_db(env, fname)[source]¶

Find database file (yml) and load its contents

Parameters

fname (str) – name of database file to load
env (InstallEnv) – installation context

Return type

Dict[str, GitProject]

Returns

registered gitprojects

pspman.serial_actions.print_prefixes(env, config=None)[source]¶

Print MetaConfig

Parameters: config (Optional[MetaConfig]) – pspman configuration

pspman.serial_actions.print_projects(env, git_projects=None, failed_projects=None)[source]¶

List all available projects

Parameters

env (InstallEnv) – Installation context
git_projects (Optional[Dict[str, GitProject]]) – projects to print
failed_projects (Optional[Dict[str, GitProject]]) – projects that have been reported to have failed

Return type

int

Returns

Error code

pspman.serial_actions.update_projects(env, git_projects, queues)[source]¶

Trigger update for all projects

Parameters

env (InstallEnv) – Installation context
git_projects (Dict[str, GitProject]) – known git projects
queues (Dict[str, PSPQueue]) – initiated queues

Return type

None

Parallel Code¶

actions¶

Parallel multiprocessed operations

All actions accept same set of args and return same type of object

Args:

env: action context

project: project on which action is requested

Returns:

project.name for indexing

project.tag feedback to update parent

success code of action to inform parent

code == 1: Successfully modified

code == -1: Failed at action

code == 0: Nothing changed

pspman.fork_actions.clone(args)[source]¶

Get (clone) the remote project.url

Parameters

args (Tuple[InstallEnv, GitProject]) –

env: installation context
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, success code of action

pspman.fork_actions.delete(args)[source]¶

Delete this project

Parameters

args (Tuple[InstallEnv, GitProject]) –

env: installation context
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, success code of action

pspman.fork_actions.failure(args)[source]¶

List failure points in projects

Parameters

args (Tuple[InstallEnv, GitProject]) –

env: installation context (ignored)
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, RET_CODE``[``fail]

pspman.fork_actions.install(args)[source]¶

Install (update) from source code.

Parameters

args (Tuple[InstallEnv, GitProject]) –

env: installation context
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, success code of action

pspman.fork_actions.success(args)[source]¶

List successful projects

Parameters

args (Tuple[InstallEnv, Optional[GitProject]]) –

env: installation context (ignored)
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, RET_CODE``[``pass]

pspman.fork_actions.update(args)[source]¶

Update (pull) source code.

Parameters

args (Tuple[InstallEnv, GitProject]) –

env: installation context
project: project to delete

Return type

Tuple[str, int, int]

Returns

project.name, project.tag, success code of action

installations¶

Parse install-config files located at standard locations to define installation methods.

Variables:: code_path: str: path to source-code prefix: str: install-prefix build_dir: str: directory to build source for installation library: str: include libraries -L include: str: include libraries -I argv: list: argv to be passed during installation
Sequence of commands called:: Installation: prefare, build, compile, install, whichever is not null Uninstallation: build, uninstall, whichever is not null

class pspman.installations.DefInstruct(instruct_file)[source]¶

Instructions definition class

indicate¶: List[str]: files that help identfy the type of install

commands¶: List[str]: commands essential for installation

type¶: str: type of installation

instruct¶: Dict[str, str]: * prepare: preparations instruction * build: build instruction * compile: compilation instruction * install: install instruction * uninstall: uninstall instruction

env¶: dict: modified environment variables

definition¶: Dict[str, Callable]: * prepare: preparations function * build: build function * compile: compilation function * install: install function * uninstall: uninstall function

Parameters

instruct_file (Path) – yaml file describing instructions

Raises

MissingInstructError – missing essential instruction
InstructFmtError – Bad format of instruct_file

instruct_def(action)[source]¶

Generate a def (function) given an instructions Object

Parameters: instructions –
Return type: Callable
Returns: Installation function constructed from action

pspman.installations.INST_METHODS = {'cmake': <pspman.installations.DefInstruct object>, 'config_make': <pspman.installations.DefInstruct object>, 'make': <pspman.installations.DefInstruct object>, 'meson': <pspman.installations.DefInstruct object>, 'pip': <pspman.installations.DefInstruct object>}¶: Methods of installation and uninstallation defined from instructions Default: make, cmake, pip, meson/ninja Extended for <instruct> by creating <config_dir>/inst_config/<instruct>.yml

class pspman.installations.Instruct(instruct_file=None)[source]¶

Format handle of yaml file

Parameters

indicate – list of file/dirnames that indicate the method
exdicate – list of file/dirnames that prohibit the method
requires – list of required dependencies to run the method
env – dict of environment variables during (un)installation
install – ordered list of steps for installation
uninstall –
ordered list of steps for uninstallation
- steps: strings that may start with a caret sign (^) ^ indicates that this step is allowed to fail
instruct_file (Optional[Path]) –

parse(instruct)[source]¶

parse data from instruct yaml file :type instruct: dict :param instruct: data out from yaml load

Raises

InstructFmtError – bad format for install instructions
MissingInstructError – essential instruction is missing

read(instruct_file)[source]¶

Read instruction file and pass data for parsing

Parameters: instruct_file (Path) – file path to <method>.yml

pspman.installations.get_instruct(config=None)[source]¶

Scan standard locations for instruction files

Parameters: custom_path – Scan this too
Return type: Dict[str, DefInstruct]

pspman.installations.run_install(i_type, code_path, prefix=<class 'pathlib.Path'>, argv=None, env=None)[source]¶

(un)Install repository

Parameters

code_path (Path) – path to source-code
prefix – --prefix flag value to be supplied
argv (Optional[List[str]]) – Arguments to be supplied during (un)installation
env (Optional[Dict[str, str]]) – Modifications in shell env variables during (un)installation

Return type

bool

Returns

False if error/failure during (un)installation, else, True

queues¶

Command Queues

class pspman.queues.CloneQueue(env, success, fail, **kwargs)[source]¶

Queue of projects to clone

on_success(project)[source]¶: run on success

class pspman.queues.DeleteQueue(env, success, fail, **kwargs)[source]¶

Queue for projects to delete

on_success(project)[source]¶: run on success

class pspman.queues.FailQueue(env, **kwargs)[source]¶: Queue to reguster Successful objects

class pspman.queues.InstallQueue(env, success, fail, **kwargs)[source]¶: Queue of projects to install

class pspman.queues.PSPQueue(env, action, fail_q=None, **kwargs)[source]¶

Base FIBO Queue object to push and retrieve tasks. File In Batch Out.

env¶: installation context

queue¶: queued GitProjects

downstream_qs¶: upstream queues feeding to this queue * success: successful projects are pushed here * fail: failed projects are pushed here

upstream_qs¶: upstream queues feeding to this queue

q_type¶: type of queue

pid¶: pid of child process

closed¶: Is the queue closed? (set by function done)

Parameters

env (InstallEnv) – installation context
action (Callable) – procedure to perform on each GitProject in the queue
items –
success_q – push to this queue, if action succeeds
fail_q (Optional[PSPQueue]) – push to this queue, if action fails
**kwargs –
- items: Optional[Dict[str, GitProject]]: initialize with items
- q_type: type of queue

add(project=None)[source]¶

Parent: Add project to queue at the end

Parameters: project (Optional[GitProject]) – project to queue
Return type: None

copy_from_client(pipe)[source]¶

Child: copy parent’s queue

Return type: bool
Returns: True if ‘close’ instruction is received

copy_to_server(project=None)[source]¶: Parent: copy to child’s queue

done(caller=None)[source]¶

Parent: No more items will be added. Finish current list and exit

Return type: None

on_failure(project)[source]¶: run on failure

on_success(project)[source]¶: run on success

run_batch()[source]¶

Child: Execute threads that run action on all items in the queue

Return type: None

start()[source]¶

Parent: Start a batch run

Returns: 0 parent: >0
Return type: child

class pspman.queues.PullQueue(env, success, fail, **kwargs)[source]¶

Queue of source codes to pull

on_success(project)[source]¶: run on success

class pspman.queues.SuccessQueue(env, **kwargs)[source]¶: Queue to reguster Successful objects

class pspman.queues.TermQueue(env, action, q_type='terminal', **kwargs)[source]¶

Terminal queues that do not have a downstream action queue

on_failure(project)[source]¶: Child: store GitProject state

on_success(project)[source]¶: Child: store GitProject state

Organisation¶

PSPManager Initialization/De-initialization¶

Aid¶

shell¶

shell functions

pspman.shell.git_clean(clone_dir, gitkwargs=None, prockwargs=None)[source]¶

Reset and clean git worktree

Parameters: clone_dir (Path) – directory in which, project is (to be) cloned
Return type: Optional[str]
Returns: Output from process_comm

pspman.shell.git_clone(clone_dir, url, name, gitkwargs=None, prockwargs=None)[source]¶

Perform a git action

Parameters

clone_dir (Path) – directory in which, project is (to be) cloned
url (str) – remote url to clone
name (str) – name (path) of project
gitkwargs (Optional[Dict[str, Optional[str]]]) – parsed from to –key[=val] and passed to git command
prockwargs (Optional[Dict[str, Any]]) – passed to process_comm

Return type

Optional[str]

Returns

Output from process_comm

pspman.shell.git_comm(cmd, g_name='process', gitkwargs=None, prockwargs=None)[source]¶

Process git-specific and subprocess-specific kwargs and run git cmd

Parameters

cmd (List[str]) – command list to run
gitkwargs (Optional[Dict[str, Optional[str]]]) – parsed from to –key[=val] and passed to git command
prockwargs (Optional[Dict[str, Any]]) – passed to process_comm

Return type

Optional[str]

Returns

Output from process_comm

pspman.shell.git_list(clone_dir, gitkwargs=None, prockwargs=None)[source]¶

Generate remote url of git

Parameters

clone_dir (Path) – directory in which, project is (to be) cloned
gitkwargs (Optional[Dict[str, Optional[str]]]) – parsed from to –key[=val] and passed to git command
prockwargs (Optional[Dict[str, Any]]) – passed to process_comm

Return type

Optional[str]

Returns

Output from process_comm

pspman.shell.git_pull(clone_dir, gitkwargs=None, prockwargs=None)[source]¶

Perform a git action

Parameters

clone_dir (Path) – directory in which, project is (to be) cloned
gitkwargs (Optional[Dict[str, Optional[str]]]) – parsed from to –key[=val] and passed to git command
prockwargs (Optional[Dict[str, Any]]) – passed to process_comm

Return type

Optional[str]

Returns

Output from process_comm

pspman.shell.process_comm(*cmd, p_name='processing', timeout=None, fail_handle='fail', **kwargs)[source]¶

Generic process definition and communication. Raw actions, outputs, errors are displayed when the parent program is called with the environment variable DEBUG = True

Parameters

*cmd – list(cmd) is passed to subprocess.Popen as first argument
p_name (str) – notified as ‘Error {p_name}: {stderr}
timeout (Optional[int]) – communicatoin timeout. If -1, ‘communicate’ isn’t called
fail – {fail,nag,report,ignore} * fail: raises CommandError * nag: Returns None, prints stderr * report: returns None, but hides stderr * ignore: returns stdout, despite error (default behaviour)
**kwargs – all are passed to subprocess.Popen

Return type

Optional[str]

Returns

stdout from command’s communication None if stderr with ‘fail == False’

Raises

CommandError –

Action Tag¶

Errors/Warnings¶

Errors¶

Exceptions, Warnings, Errors

exception pspman.errors.ClosedQueueError(queue)[source]¶

Operation can’t be performed, since the queue has been closed

Parameters: queue – PSPQueue: the queue that threw the error

exception pspman.errors.CommandError(cmd, err=None)[source]¶

Base class for subprocess failure

Parameters

cmd (list) – command passed to shell for execution
err (Optional[str]) – stderr received from shell after failure

exception pspman.errors.GitURLError[source]¶: Git URL (and hence or otherwise, name) can’t be determined

exception pspman.errors.InstructError[source]¶: Base class for instruction files error

exception pspman.errors.InstructFmtError(instruct_file)[source]¶

Error parsing installation instructions file

Parameters: instruct_file (Union[str, PathLike]) – name of file with fault

exception pspman.errors.InstructTypeError(key, found_type, want_type)[source]¶

Wrong type of instruction

Parameters

key (str) – variable with bad type
found_type (type) – type of supplied variable
want_type (str) – type expected

exception pspman.errors.MissingInstructError(missing)[source]¶

Error in installation instructions file: necessary instruction missing

Parameters: missing (str) – name of argument missing

exception pspman.errors.PSPManError[source]¶: base error for papman

exception pspman.errors.PathNameError(*args)[source]¶: Path/URL (and hence or otherwise, name) can’t be determined