Skip to content

Releases: BlockScience/koi-net

v1.3.0-beta.2

05 Jan 22:22

Choose a tag to compare

v1.3.0-beta.2 Pre-release
Pre-release
  • added "READY" stdout signal for CLI orchestration
  • improved node and module errors
  • moved module ref and get module functions to utils.py
  • added sync command replacing install
  • moved list-nodes to node list and list-node-types to node modules
  • added wipe cache command
  • start node function now waits for READY signal
  • added _container field in node assembler to allow overrides
  • added config_only flag - will likely remove later
  • add log error for PEM decrypt failure
  • updated control loop component to handle both startup and shutdown events
  • base assembly and node container initialize these events and set/await them
  • unified and downsized node and network cli commands
  • added node module interface which exposes a simple standalone CLI for node modules
  • network interface significantly lighter with focus on configuration
  • node interface refactored as a wrapper for the node module interface called through the CLI, resolves conflicts from set up within the same python environment - includes "STOP" and "READY" process signalling
  • added mutate context manager to config loader
  • excluding env variables from yaml dump when saving
  • added kwarg only barrier to kobjqueue.push method to prevent unexpected behavior
  • moved string literals into constants
  • added newline prefix to READY signal to avoid buffer flushing issues
  • overhauled commands - much simpler wrapper for node and network interfaces
  • much more comprehensive and robust run methods for node and network interfaces
  • better handling of stdio interfaces and buffers, general clean up
  • add type checking imports to interface.py, not sure if it actually spins up faster

v1.3.0-beta.1

29 Dec 17:04

Choose a tag to compare

v1.3.0-beta.1 Pre-release
Pre-release
  • added new CLI tool for managing nodes and networks, mainly intended for easier maintenance of local development environments
  • renamed NodeConfig -> BaseNodeConfig and renamed config.core -> config.base
  • split sub configs into separate files: koi_net_config and env_config
  • refactored env_config to use pydantic's base settings module which has much better handling of environment variables
  • updated create function testing out smart env variable detection
  • added port_acquire method to server component, can automatically reassign its own port to avoid conflicts with other nodes running locally
  • changed env config to a Field with a default factory to avoid errors on library import
  • added config.core back for backwards compatibility
  • improved list-nodes and create commands with better error message and config handling
  • added delay in rotating file handler so it doesn't create empty logs
  • more comprehensive tracking and set up for nodes and network configuration
  • added mutate_config context manager for nodes
  • added set first contact command to update network and individual node configs
  • added setattr method to config proxy to fix writing const vars
  • updated network start method and command
  • generalized proxy and loader solution to both node and network config
  • no longer setting first contact of the first contact to be itself
  • moved modules to commands and interfaces submodules
  • added aliases in get node modules method
  • better internal handling for node interface methods
  • log system added back as a component, initialized on node class init so it doesn't pollute the cwd during import, also refactored to allow config params to be passed in to set which handlers to use etc., currently kwargs passed through from node assembler init
  • server now executes in a thread, no longer main thread 'entrypoint' by inheriting from uvicorn.Server
  • added BaseAssembly class which blocks on a new shutdown threading event rather than using the main thread
  • added a control loop component which sets the shutdown event when it receives "STOP" via stdin
  • added new default stop method in threadworker class which joins if the thread was started, updated event and kobj worker accordingly
  • delete node command now cascades to first contact in network config
  • updated in_directory decorator to use contextlib.chdir, new default supress output option for node subprocesses
  • updated node stop command to use stdin
  • added verbose flag
  • added keyboard interrupt for node container
  • refactored poller as thread, removed entrypoint parent class
  • better get_node_modules implementation includes non entrypoints
  • better poller implementation doesn't hang as long when shutting down
  • version bump 1.3.0-beta.1, removed build system and dev dependencies

v1.2.4

12 Dec 18:31
1ca654b

Choose a tag to compare

  • changed all absolute koi_net imports to relative imports
  • better handling errors in request handler, comprehensive coverage and grouping for easier exception handling
  • fixed mislabeled exceptions in server
  • added better exception documentation
  • factored out ErrorResponse from internal systems
  • when peer node responds with an error, the response payload is instead validated and the corresponding exception is raised, simplifying internal code and allow exception handling by group (i.e. any request error)
  • refactored sync manager and resolver to use try.. except.. rather than checking if the response type is an error response
  • added a safe_flush method to the event buffer, now used by the event worker to automatically roll back changes when exceptions occur, improved error handling
  • added force_flush on event worker shut down so thread doesn't hang on unreachable nodes
  • added report exception wrapper to automatically log warnings for all request error exceptions when make_request is called
  • handled all request error exceptions in functions calling make_request, and everything upstream of them
  • fixed improper exception import, added log warning for event buffer reset

v1.2.3

10 Dec 15:18
e493fa5

Choose a tag to compare

  • better exception handling
  • removed all generic Exception calls
  • unified all custom errors deriving from KoiNetError defined in koi_net.exceptions
  • dropped rich as a dependency

v1.2.2

10 Dec 04:55

Choose a tag to compare

  • converted HandlerContext to a dataclass for easier extensibility

v1.2.1

05 Dec 20:28
cffa0e8

Choose a tag to compare

  • moved logging system out of node container into module init so logging format is configured immediately
  • removed print statements from build system and added appropriate log statements
  • updated component start/stop logging

v1.2.0

03 Dec 22:58
804402a

Choose a tag to compare

  • restructured code base for a better "manual dependency injection" approach, node implementations are classes now, not instances
  • factored effector out of core subsystems to remove dependency loops
  • created knowledge and event processing threads to handle decoupling via knowledge object and event queues
  • created node assembler and node container classes for building and storing node components respectively, node assembler can be configured to override subsystem classes
  • knowledge handlers passed directly into node assembler now
  • added "RID types of interest" field to config (defaults to orn:koi-net.node), which informs the node contact handler, automatically subscribing too and backfilling from node providers of those types
  • handshake is now its own component
  • created JSON schemas for all protocol objects
  • better conditions for determining when no coordinator is known
  • event worker flushes buffers before shutting down
  • experimental CLI to install, create, and run node implementations
  • set up structlog, moving towards better logging practices
  • removed behaviors
  • moved logic into lifecycle
  • added effector back to handler context
  • factored out action context for now
  • finally added support for "limit" in poll events request
  • added ApiModels as a discriminated union using type consts in the request/payload models
  • added a model map linking API paths (aka methods) to request and response models (both raw and enveloped versions) -- refactored request handler accordingly
  • major overhaul moving a lot of logic from server -> response_handler: wraps with envelope validation into a single method
  • entrypoints (poller, server) should be largely interfaces not contain much logic themselves
  • working on a better dependency injection system
  • moved around some default values and dependencies
  • figured out clean unified design for assembler and blueprints, works like a regular class is expected to
  • refined node assembler, parses through all base classes in reverse order to construct _build_order
  • moved base, partial, and full node definitions back to core.py
  • attempt to refactor config system
  • separate subsystem ConfigLoader handles reading and writing to YAML
  • config components now separated by full/partial node
  • generating on missing logic moved to model validators,
  • added support for non factories to dependency injection framework -- detects non callables, and Pydantic BaseModels and treats them "as is"
  • restructuring and renaming
  • queues and buffer use 'push' instead of redundant push_kobj/push_event_to
  • moved config system into a node component
  • removed old logging system from example nodes
  • updated typing and error logging
  • added LNAV config file for koi-net ndjson logs
  • removed cli tools, removed cli entrypoint in pyproject, moving to 'cli' branch, won't be a part of v1.2
  • fixing coordinator catch up, no longer tries to query partial nodes or nodes that don't provide orn:koi-net.node state
  • moved catch up function in lifecycle to a sync manager component
  • decomposed assembler build function into three steps
  • added a dependency graph visualizer
  • updated LNAV config
  • cleaned up knowledge handler class
  • added doc strings to assembler
  • pass on config modules - improved logging and docs
  • update documentation, logging, organization from entry point components
  • cleaning up network methods, documentation and small code improvements
  • added a reset for the node network timeout
  • updated docstrings and cleaned up code in processor modules
  • updated documentation for protocol and worker modules
  • generalized poll event buffer, to event buffer including timing features for broadcast batching
  • added broadcast event buffer as an independent component
  • moved internal config for workers into config class
  • removed unused default effector actions
  • renamed secure to secure manager
  • restructured sync manager and catch up call in lifecycle
  • refactored effector to work similar to the processing pipeline, accepts a list of deref handlers
  • made public key overwrite warning more specific
  • gave effector deref handlers access to handler context, readded post init injection to effector
  • hacked type annotation return from node assembler to give a better dynamic representation of the node container to the type checker
  • added write through option on deref function
  • spun out config proxy class from config loader
  • renamed config cls to config schema
  • better separation of concerns for config system
  • renamed component config_cls -> config_schema
  • switched assembler functions back to chain calls
  • implemented cache directly with config dependency (necessary for new config system)
  • moved all code execution out of init functions
  • lifecycle start now calls config loader, and loads priv key to secure manager
  • removed lifespan from fastapi app, now done using lifecycle context manager in server class to allow for config loading before uvicorn starts
  • added kobj worker config
  • automated component build order via kahn's algorithm on dependency graph
  • working on cycle detection and sending useful dev errors
  • moved start / stop logic into worker classes
  • factored out lifecycle component, instead the node container object will automatically run start() and stop() functions for components which provide them
  • config now loads at init time, not at startup
  • entrypoint components run() method treated as special main thread function by container
  • refactored assembly mechanism: new assembly artifact class does most of the assembly work, built once for each node assembly
  • node assembly builds components and passes them along with the artifact into a new node container class
  • node container class has start, stop, and run functions, they will automatically turn components on in order and then run the entrypoint
  • experimenting with explicit overrides, cache schema has a component.static decorator now
  • added a consts module
  • moved assembler, component, container, and consts to shared build module, cleaned up documentation
  • moved handshaker, profile monitor (renamed from self start), and sync manager to behaviors module
  • removed catcher upper component and split its start up functions between the handshaker and sync manager
  • cleaning up, added get_logger method in log system, may have components import log as a dependency
  • renamed assembly artifact -> build artifact
  • added a new comp_order decorator to give start and stop orderer more information
  • comp_order.worker starts worker components before and stops them after all other components
  • renamed component -> comp_type
  • removed factory (for now), default comp type is singleton
  • moved build_app() call to init of server, not run method
  • added comp_order.worker to base worker class
  • readded access log to uvicorn
  • added more specific typehints for full node and partial node config
  • updated log system
  • achieved compatibility with uvicorn logs including errors and access!
  • handshaker bug fix, handshakes when there aren't any neighbors

v1.2.0-beta.9

03 Dec 22:41

Choose a tag to compare

v1.2.0-beta.9 Pre-release
Pre-release
  • readded access log to uvicorn
  • added more specific typehints for full node and partial node config
  • updated log system
  • achieved compatibility with uvicorn logs including errors and access!

v1.2.0-beta.8

03 Dec 19:33

Choose a tag to compare

v1.2.0-beta.8 Pre-release
Pre-release
  • renamed assembly artifact -> build artifact
  • added a new comp_order decorator to give start and stop orderer more information
  • comp_order.worker starts worker components before and stops them after all other components
  • renamed component -> comp_type
  • removed factory (for now), default comp type is singleton
  • moved build_app() call to init of server, not run method
  • added comp_order.worker to base worker class

v1.2.0-beta.7

03 Dec 04:51

Choose a tag to compare

v1.2.0-beta.7 Pre-release
Pre-release
  • moved start / stop logic into worker classes
  • factored out lifecycle component, instead the node container object will automatically run start() and stop() functions for components which provide them
  • config now loads at init time, not at startup
  • entrypoint components run() method treated as special main thread function by container
  • refactored assembly mechanism: new assembly artifact class does most of the assembly work, built once for each node assembly
  • node assembly builds components and passes them along with the artifact into a new node container class
  • node container class has start, stop, and run functions, they will automatically turn components on in order and then run the entrypoint
  • experimenting with explicit overrides, cache schema has a component.static decorator now
  • added a consts module
  • moved assembler, component, container, and consts to shared build module, cleaned up documentation
  • moved handshaker, profile monitor (renamed from self start), and sync manager to behaviors module
  • removed catcher upper component and split its start up functions between the handshaker and sync manager
  • cleaning up, added get_logger method in log system, may have components import log as a dependency