Create a Module

Before creating a module, we will need to go over the following:

• Module Path is the path from the root directory in commune to a specific module. These module paths are used to reference modules easily using a string. For instance, if a module is place in commune/model/llm, then its module path is model.llm
• Module Tree is a collection of all of the module paths in your local folder. This forms a tree that is defined by the user

If a module is defined outside of the commune directory, then it will not be recorded in the module tree, but can still be used with most of the commune.Modules functions that are not related to the module tree.

Way 1: Config and Python Class

A module can be instantiated as a config paired with a python file. These are both contained within a folder as follows. In this example, we are following the buzz train and creating the Nth LLM.

# Module path of model.llm
commune\
  model\
    llm\
      llm_model.py
      llm_model.yaml

This creates a module path that represents the module path from commune, so in this case it would be model.llm

The config and python class are as follows.

"model": "text-davinci-003"
"tokenizer": "gpt2"

import commune as c
class LLM(c.Module):
  def __init__(self,config=None):
    # set the config , if None, use the default yaml
    self.set_config(config)

    # do whatever the fuck else you want
    self.set_tokenizer(self.config.tokenizer)
    self.set_model(self.config.model)
  def set_model(model:str):
    ...
  def set_tokenizer(tokenizer: str):
    ...

Way 2: Python only, no config

Now lets define the same module without a config for those that hate configs.

# Module path of model.llm
commune\
  model\
    llm.py

With the python file being

import commune as c
class LLM(c.Module):
  def __init__(self, model='text-davinci-003', tokenizer='gpt2'):
    # note how we dont need to set the config
    self.set_tokenizer(tokenizer)
    self.set_model(model)
  def set_model(model:str):
    ...
  def set_tokenizer(tokenizer: str):
    ...

If you want your module to be contianed within a directory without affecting the module path