Update mach from gecko tree

python/mach/README.rst

@@ -10,319 +10,4 @@ executable *driver* script (named whatever you want), and write mach
 commands. When the *driver* is executed, mach dispatches to the
 requested command handler automatically.
 
-Features
+To learn more, read the docs in ``docs/``.
-========
-
-On a high level, mach is similar to using argparse with subparsers (for
-command handling). When you dig deeper, mach offers a number of
-additional features:
-
-Distributed command definitions
-  With optparse/argparse, you have to define your commands on a central
-  parser instance. With mach, you annotate your command methods with
-  decorators and mach finds and dispatches to them automatically.
-
-Command categories
-  Mach commands can be grouped into categories when displayed in help.
-  This is currently not possible with argparse.
-
-Logging management
-  Mach provides a facility for logging (both classical text and
-  structured) that is available to any command handler.
-
-Settings files
-  Mach provides a facility for reading settings from an ini-like file
-  format.
-
-Components
-==========
-
-Mach is conceptually composed of the following components:
-
-core
-  The mach core is the core code powering mach. This is a Python package
-  that contains all the business logic that makes mach work. The mach
-  core is common to all mach deployments.
-
-commands
-  These are what mach dispatches to. Commands are simply Python methods
-  registered as command names. The set of commands is unique to the
-  environment mach is deployed in.
-
-driver
-  The *driver* is the entry-point to mach. It is simply an executable
-  script that loads the mach core, tells it where commands can be found,
-  then asks the mach core to handle the current request. The driver is
-  unique to the deployed environment. But, it's usually based on an
-  example from this source tree.
-
-Project State
-=============
-
-mach was originally written as a command dispatching framework to aid
-Firefox development. While the code is mostly generic, there are still
-some pieces that closely tie it to Mozilla/Firefox. The goal is for
-these to eventually be removed and replaced with generic features so
-mach is suitable for anybody to use. Until then, mach may not be the
-best fit for you.
-
-Implementing Commands
----------------------
-
-Mach commands are defined via Python decorators.
-
-All the relevant decorators are defined in the *mach.decorators* module.
-The important decorators are as follows:
-
-CommandProvider
-  A class decorator that denotes that a class contains mach
-  commands. The decorator takes no arguments.
-
-Command
-  A method decorator that denotes that the method should be called when
-  the specified command is requested. The decorator takes a command name
-  as its first argument and a number of additional arguments to
-  configure the behavior of the command.
-
-CommandArgument
-  A method decorator that defines an argument to the command. Its
-  arguments are essentially proxied to ArgumentParser.add_argument()
-
-Classes with the *@CommandProvider* decorator *must* have an *__init__*
-method that accepts 1 or 2 arguments. If it accepts 2 arguments, the
-2nd argument will be a *MachCommandContext* instance. This is just a named
-tuple containing references to objects provided by the mach driver.
-
-Here is a complete example::
-
-    from mach.decorators import (
-        CommandArgument,
-        CommandProvider,
-        Command,
-    )
-
-    @CommandProvider
-    class MyClass(object):
-        @Command('doit', help='Do ALL OF THE THINGS.')
-        @CommandArgument('--force', '-f', action='store_true',
-            help='Force doing it.')
-        def doit(self, force=False):
-            # Do stuff here.
-
-When the module is loaded, the decorators tell mach about all handlers.
-When mach runs, it takes the assembled metadata from these handlers and
-hooks it up to the command line driver. Under the hood, arguments passed
-to the decorators are being used to help mach parse command arguments,
-formulate arguments to the methods, etc. See the documentation in the
-*mach.base* module for more.
-
-The Python modules defining mach commands do not need to live inside the
-main mach source tree.
-
-Conditionally Filtering Commands
---------------------------------
-
-Sometimes it might only make sense to run a command given a certain
-context. For example, running tests only makes sense if the product
-they are testing has been built, and said build is available. To make
-sure a command is only runnable from within a correct context, you can
-define a series of conditions on the *Command* decorator.
-
-A condition is simply a function that takes an instance of the
-*CommandProvider* class as an argument, and returns True or False. If
-any of the conditions defined on a command return False, the command
-will not be runnable. The doc string of a condition function is used in
-error messages, to explain why the command cannot currently be run.
-
-Here is an example:
-
-    from mach.decorators import (
-        CommandProvider,
-        Command,
-    )
-
-    def build_available(cls):
-        """The build needs to be available."""
-        return cls.build_path is not None
-
-    @CommandProvider
-    class MyClass(MachCommandBase):
-        def __init__(self, build_path=None):
-            self.build_path = build_path
-
-        @Command('run_tests', conditions=[build_available])
-        def run_tests(self):
-            # Do stuff here.
-
-It is important to make sure that any state needed by the condition is
-available to instances of the command provider.
-
-By default all commands without any conditions applied will be runnable,
-but it is possible to change this behaviour by setting *require_conditions*
-to True:
-
-    m = mach.main.Mach()
-    m.require_conditions = True
-
-Minimizing Code in Commands
----------------------------
-
-Mach command modules, classes, and methods work best when they are
-minimal dispatchers. The reason is import bloat. Currently, the mach
-core needs to import every Python file potentially containing mach
-commands for every command invocation. If you have dozens of commands or
-commands in modules that import a lot of Python code, these imports
-could slow mach down and waste memory.
-
-It is thus recommended that mach modules, classes, and methods do as
-little work as possible. Ideally the module should only import from
-the *mach* package. If you need external modules, you should import them
-from within the command method.
-
-To keep code size small, the body of a command method should be limited
-to:
-
-1. Obtaining user input (parsing arguments, prompting, etc)
-2. Calling into some other Python package
-3. Formatting output
-
-Of course, these recommendations can be ignored if you want to risk
-slower performance.
-
-In the future, the mach driver may cache the dispatching information or
-have it intelligently loaded to facilitate lazy loading.
-
-Logging
-=======
-
-Mach configures a built-in logging facility so commands can easily log
-data.
-
-What sets the logging facility apart from most loggers you've seen is
-that it encourages structured logging. Instead of conventional logging
-where simple strings are logged, the internal logging mechanism logs all
-events with the following pieces of information:
-
-* A string *action*
-* A dict of log message fields
-* A formatting string
-
-Essentially, instead of assembling a human-readable string at
-logging-time, you create an object holding all the pieces of data that
-will constitute your logged event. For each unique type of logged event,
-you assign an *action* name.
-
-Depending on how logging is configured, your logged event could get
-written a couple of different ways.
-
-JSON Logging
-------------
-
-Where machines are the intended target of the logging data, a JSON
-logger is configured. The JSON logger assembles an array consisting of
-the following elements:
-
-* Decimal wall clock time in seconds since UNIX epoch
-* String *action* of message
-* Object with structured message data
-
-The JSON-serialized array is written to a configured file handle.
-Consumers of this logging stream can just perform a readline() then feed
-that into a JSON deserializer to reconstruct the original logged
-message. They can key off the *action* element to determine how to
-process individual events. There is no need to invent a parser.
-Convenient, isn't it?
-
-Logging for Humans
-------------------
-
-Where humans are the intended consumer of a log message, the structured
-log message are converted to more human-friendly form. This is done by
-utilizing the *formatting* string provided at log time. The logger
-simply calls the *format* method of the formatting string, passing the
-dict containing the message's fields.
-
-When *mach* is used in a terminal that supports it, the logging facility
-also supports terminal features such as colorization. This is done
-automatically in the logging layer - there is no need to control this at
-logging time.
-
-In addition, messages intended for humans typically prepends every line
-with the time passed since the application started.
-
-Logging HOWTO
--------------
-
-Structured logging piggybacks on top of Python's built-in logging
-infrastructure provided by the *logging* package. We accomplish this by
-taking advantage of *logging.Logger.log()*'s *extra* argument. To this
-argument, we pass a dict with the fields *action* and *params*. These
-are the string *action* and dict of message fields, respectively. The
-formatting string is passed as the *msg* argument, like normal.
-
-If you were logging to a logger directly, you would do something like:
-
-    logger.log(logging.INFO, 'My name is {name}',
-        extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
-
-The JSON logging would produce something like:
-
-    [1339985554.306338, "my_name", {"name": "Gregory"}]
-
-Human logging would produce something like:
-
-     0.52 My name is Gregory
-
-Since there is a lot of complexity using logger.log directly, it is
-recommended to go through a wrapping layer that hides part of the
-complexity for you. The easiest way to do this is by utilizing the
-LoggingMixin:
-
-    import logging
-    from mach.mixin.logging import LoggingMixin
-
-    class MyClass(LoggingMixin):
-        def foo(self):
-             self.log(logging.INFO, 'foo_start', {'bar': True},
-                 'Foo performed. Bar: {bar}')
-
-Entry Points
-============
-
-It is possible to use setuptools' entry points to load commands
-directly from python packages. A mach entry point is a function which
-returns a list of files or directories containing mach command
-providers. e.g.::
-
-    def list_providers():
-        providers = []
-        here = os.path.abspath(os.path.dirname(__file__))
-        for p in os.listdir(here):
-            if p.endswith('.py'):
-                providers.append(os.path.join(here, p))
-        return providers
-
-See http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
-for more information on creating an entry point. To search for entry
-point plugins, you can call *load_commands_from_entry_point*. This
-takes a single parameter called *group*. This is the name of the entry
-point group to load and defaults to ``mach.providers``. e.g.::
-
-    mach.load_commands_from_entry_point("mach.external.providers")
-
-Adding Global Arguments
-=======================
-
-Arguments to mach commands are usually command-specific. However,
-mach ships with a handful of global arguments that apply to all
-commands.
-
-It is possible to extend the list of global arguments. In your
-*mach driver*, simply call ``add_global_argument()`` on your
-``mach.main.Mach`` instance. e.g.::
-
-   mach = mach.main.Mach(os.getcwd())
-
-   # Will allow --example to be specified on every mach command.
-   mach.add_global_argument('--example', action='store_true',
-       help='Demonstrate an example global argument.')

python/mach/docs/commands.rst

@@ -0,0 +1,145 @@
+.. _mach_commands:
+
+=====================
+Implementing Commands
+=====================
+
+Mach commands are defined via Python decorators.
+
+All the relevant decorators are defined in the *mach.decorators* module.
+The important decorators are as follows:
+
+:py:func:`CommandProvider <mach.decorators.CommandProvider>`
+  A class decorator that denotes that a class contains mach
+  commands. The decorator takes no arguments.
+
+:py:func:`Command <mach.decorators.Command>`
+  A method decorator that denotes that the method should be called when
+  the specified command is requested. The decorator takes a command name
+  as its first argument and a number of additional arguments to
+  configure the behavior of the command.
+
+:py:func:`CommandArgument <mach.decorators.CommandArgument>`
+  A method decorator that defines an argument to the command. Its
+  arguments are essentially proxied to ArgumentParser.add_argument()
+
+:py:func:`SubCommand <mach.decorators.SubCommand>`
+  A method decorator that denotes that the method should be a
+  sub-command to an existing ``@Command``. The decorator takes the
+  parent command name as its first argument and the sub-command name
+  as its second argument.
+
+  ``@CommandArgument`` can be used on ``@SubCommand`` instances just
+  like they can on ``@Command`` instances.
+
+Classes with the ``@CommandProvider`` decorator **must** have an
+``__init__`` method that accepts 1 or 2 arguments. If it accepts 2
+arguments, the 2nd argument will be a
+:py:class:`mach.base.CommandContext` instance.
+
+Here is a complete example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       CommandArgument,
+       CommandProvider,
+       Command,
+   )
+
+   @CommandProvider
+   class MyClass(object):
+       @Command('doit', help='Do ALL OF THE THINGS.')
+       @CommandArgument('--force', '-f', action='store_true',
+           help='Force doing it.')
+       def doit(self, force=False):
+           # Do stuff here.
+
+When the module is loaded, the decorators tell mach about all handlers.
+When mach runs, it takes the assembled metadata from these handlers and
+hooks it up to the command line driver. Under the hood, arguments passed
+to the decorators are being used to help mach parse command arguments,
+formulate arguments to the methods, etc. See the documentation in the
+:py:mod:`mach.base` module for more.
+
+The Python modules defining mach commands do not need to live inside the
+main mach source tree.
+
+Conditionally Filtering Commands
+================================
+
+Sometimes it might only make sense to run a command given a certain
+context. For example, running tests only makes sense if the product
+they are testing has been built, and said build is available. To make
+sure a command is only runnable from within a correct context, you can
+define a series of conditions on the
+:py:func:`Command <mach.decorators.Command>` decorator.
+
+A condition is simply a function that takes an instance of the
+:py:func:`mach.decorators.CommandProvider` class as an argument, and
+returns ``True`` or ``False``. If any of the conditions defined on a
+command return ``False``, the command will not be runnable. The
+docstring of a condition function is used in error messages, to explain
+why the command cannot currently be run.
+
+Here is an example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       CommandProvider,
+       Command,
+   )
+
+   def build_available(cls):
+       """The build needs to be available."""
+       return cls.build_path is not None
+
+    @CommandProvider
+   class MyClass(MachCommandBase):
+       def __init__(self, build_path=None):
+           self.build_path = build_path
+
+       @Command('run_tests', conditions=[build_available])
+       def run_tests(self):
+           # Do stuff here.
+
+It is important to make sure that any state needed by the condition is
+available to instances of the command provider.
+
+By default all commands without any conditions applied will be runnable,
+but it is possible to change this behaviour by setting
+``require_conditions`` to ``True``:
+
+.. code-block:: python
+
+   m = mach.main.Mach()
+   m.require_conditions = True
+
+Minimizing Code in Commands
+===========================
+
+Mach command modules, classes, and methods work best when they are
+minimal dispatchers. The reason is import bloat. Currently, the mach
+core needs to import every Python file potentially containing mach
+commands for every command invocation. If you have dozens of commands or
+commands in modules that import a lot of Python code, these imports
+could slow mach down and waste memory.
+
+It is thus recommended that mach modules, classes, and methods do as
+little work as possible. Ideally the module should only import from
+the :py:mod:`mach` package. If you need external modules, you should
+import them from within the command method.
+
+To keep code size small, the body of a command method should be limited
+to:
+
+1. Obtaining user input (parsing arguments, prompting, etc)
+2. Calling into some other Python package
+3. Formatting output
+
+Of course, these recommendations can be ignored if you want to risk
+slower performance.
+
+In the future, the mach driver may cache the dispatching information or
+have it intelligently loaded to facilitate lazy loading.

python/mach/docs/driver.rst

@@ -0,0 +1,51 @@
+.. _mach_driver:
+
+=======
+Drivers
+=======
+
+Entry Points
+============
+
+It is possible to use setuptools' entry points to load commands
+directly from python packages. A mach entry point is a function which
+returns a list of files or directories containing mach command
+providers. e.g.:
+
+.. code-block:: python
+
+   def list_providers():
+       providers = []
+       here = os.path.abspath(os.path.dirname(__file__))
+       for p in os.listdir(here):
+           if p.endswith('.py'):
+               providers.append(os.path.join(here, p))
+       return providers
+
+See http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
+for more information on creating an entry point. To search for entry
+point plugins, you can call
+:py:meth:`mach.main.Mach.load_commands_from_entry_point`. e.g.:
+
+.. code-block:: python
+
+   mach.load_commands_from_entry_point("mach.external.providers")
+
+Adding Global Arguments
+=======================
+
+Arguments to mach commands are usually command-specific. However,
+mach ships with a handful of global arguments that apply to all
+commands.
+
+It is possible to extend the list of global arguments. In your
+*mach driver*, simply call
+:py:meth:`mach.main.Mach.add_global_argument`. e.g.:
+
+.. code-block:: python
+
+   mach = mach.main.Mach(os.getcwd())
+
+   # Will allow --example to be specified on every mach command.
+   mach.add_global_argument('--example', action='store_true',
+       help='Demonstrate an example global argument.')

python/mach/docs/index.rst

@@ -0,0 +1,74 @@
+====
+mach
+====
+
+Mach (German for *do*) is a generic command dispatcher for the command
+line.
+
+To use mach, you install the mach core (a Python package), create an
+executable *driver* script (named whatever you want), and write mach
+commands. When the *driver* is executed, mach dispatches to the
+requested command handler automatically.
+
+Features
+========
+
+On a high level, mach is similar to using argparse with subparsers (for
+command handling). When you dig deeper, mach offers a number of
+additional features:
+
+Distributed command definitions
+  With optparse/argparse, you have to define your commands on a central
+  parser instance. With mach, you annotate your command methods with
+  decorators and mach finds and dispatches to them automatically.
+
+Command categories
+  Mach commands can be grouped into categories when displayed in help.
+  This is currently not possible with argparse.
+
+Logging management
+  Mach provides a facility for logging (both classical text and
+  structured) that is available to any command handler.
+
+Settings files
+  Mach provides a facility for reading settings from an ini-like file
+  format.
+
+Components
+==========
+
+Mach is conceptually composed of the following components:
+
+core
+  The mach core is the core code powering mach. This is a Python package
+  that contains all the business logic that makes mach work. The mach
+  core is common to all mach deployments.
+
+commands
+  These are what mach dispatches to. Commands are simply Python methods
+  registered as command names. The set of commands is unique to the
+  environment mach is deployed in.
+
+driver
+  The *driver* is the entry-point to mach. It is simply an executable
+  script that loads the mach core, tells it where commands can be found,
+  then asks the mach core to handle the current request. The driver is
+  unique to the deployed environment. But, it's usually based on an
+  example from this source tree.
+
+Project State
+=============
+
+mach was originally written as a command dispatching framework to aid
+Firefox development. While the code is mostly generic, there are still
+some pieces that closely tie it to Mozilla/Firefox. The goal is for
+these to eventually be removed and replaced with generic features so
+mach is suitable for anybody to use. Until then, mach may not be the
+best fit for you.
+
+.. toctree::
+   :maxdepth: 1
+
+   commands
+   driver
+   logging

python/mach/docs/logging.rst

@@ -0,0 +1,100 @@
+.. _mach_logging:
+
+=======
+Logging
+=======
+
+Mach configures a built-in logging facility so commands can easily log
+data.
+
+What sets the logging facility apart from most loggers you've seen is
+that it encourages structured logging. Instead of conventional logging
+where simple strings are logged, the internal logging mechanism logs all
+events with the following pieces of information:
+
+* A string *action*
+* A dict of log message fields
+* A formatting string
+
+Essentially, instead of assembling a human-readable string at
+logging-time, you create an object holding all the pieces of data that
+will constitute your logged event. For each unique type of logged event,
+you assign an *action* name.
+
+Depending on how logging is configured, your logged event could get
+written a couple of different ways.
+
+JSON Logging
+============
+
+Where machines are the intended target of the logging data, a JSON
+logger is configured. The JSON logger assembles an array consisting of
+the following elements:
+
+* Decimal wall clock time in seconds since UNIX epoch
+* String *action* of message
+* Object with structured message data
+
+The JSON-serialized array is written to a configured file handle.
+Consumers of this logging stream can just perform a readline() then feed
+that into a JSON deserializer to reconstruct the original logged
+message. They can key off the *action* element to determine how to
+process individual events. There is no need to invent a parser.
+Convenient, isn't it?
+
+Logging for Humans
+==================
+
+Where humans are the intended consumer of a log message, the structured
+log message are converted to more human-friendly form. This is done by
+utilizing the *formatting* string provided at log time. The logger
+simply calls the *format* method of the formatting string, passing the
+dict containing the message's fields.
+
+When *mach* is used in a terminal that supports it, the logging facility
+also supports terminal features such as colorization. This is done
+automatically in the logging layer - there is no need to control this at
+logging time.
+
+In addition, messages intended for humans typically prepends every line
+with the time passed since the application started.
+
+Logging HOWTO
+=============
+
+Structured logging piggybacks on top of Python's built-in logging
+infrastructure provided by the *logging* package. We accomplish this by
+taking advantage of *logging.Logger.log()*'s *extra* argument. To this
+argument, we pass a dict with the fields *action* and *params*. These
+are the string *action* and dict of message fields, respectively. The
+formatting string is passed as the *msg* argument, like normal.
+
+If you were logging to a logger directly, you would do something like:
+
+.. code-block:: python
+
+   logger.log(logging.INFO, 'My name is {name}',
+       extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
+
+The JSON logging would produce something like::
+
+   [1339985554.306338, "my_name", {"name": "Gregory"}]
+
+Human logging would produce something like::
+
+   0.52 My name is Gregory
+
+Since there is a lot of complexity using logger.log directly, it is
+recommended to go through a wrapping layer that hides part of the
+complexity for you. The easiest way to do this is by utilizing the
+LoggingMixin:
+
+.. code-block:: python
+
+   import logging
+   from mach.mixin.logging import LoggingMixin
+
+   class MyClass(LoggingMixin):
+       def foo(self):
+            self.log(logging.INFO, 'foo_start', {'bar': True},
+                'Foo performed. Bar: {bar}')

python/mach/mach/base.py

@@ -2,7 +2,7 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 
 class CommandContext(object):
@@ -44,67 +44,3 @@ class UnrecognizedArgumentError(MachError):
 
         self.command = command
         self.arguments = arguments
-
-
-class MethodHandler(object):
-    """Describes a Python method that implements a mach command.
-
-    Instances of these are produced by mach when it processes classes
-    defining mach commands.
-    """
-    __slots__ = (
-        # The Python class providing the command. This is the class type not
-        # an instance of the class. Mach will instantiate a new instance of
-        # the class if the command is executed.
-        'cls',
-
-        # Whether the __init__ method of the class should receive a mach
-        # context instance. This should only affect the mach driver and how
-        # it instantiates classes.
-        'pass_context',
-
-        # The name of the method providing the command. In other words, this
-        # is the str name of the attribute on the class type corresponding to
-        # the name of the function.
-        'method',
-
-        # The name of the command.
-        'name',
-
-        # String category this command belongs to.
-        'category',
-
-        # Description of the purpose of this command.
-        'description',
-
-        # Functions used to 'skip' commands if they don't meet the conditions
-        # in a given context.
-        'conditions',
-
-        # argparse.ArgumentParser instance to use as the basis for command
-        # arguments.
-        'parser',
-
-        # Arguments added to this command's parser. This is a 2-tuple of
-        # positional and named arguments, respectively.
-        'arguments',
-
-        # Argument groups added to this command's parser.
-        'argument_group_names',
-    )
-
-    def __init__(self, cls, method, name, category=None, description=None,
-        conditions=None, parser=None, arguments=None,
-        argument_group_names=None, pass_context=False):
-
-        self.cls = cls
-        self.method = method
-        self.name = name
-        self.category = category
-        self.description = description
-        self.conditions = conditions or []
-        self.parser = parser
-        self.arguments = arguments or []
-        self.argument_group_names = argument_group_names or []
-        self.pass_context = pass_context
-

python/mach/mach/commands/commandinfo.py

@@ -2,11 +2,12 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, # You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import print_function, unicode_literals
+from __future__ import absolute_import, print_function, unicode_literals
 
 from mach.decorators import (
     CommandProvider,
     Command,
+    CommandArgument,
 )
 
 
@@ -22,11 +23,16 @@ class BuiltinCommands(object):
 
     @Command('mach-debug-commands', category='misc',
         description='Show info about available mach commands.')
-    def debug_commands(self):
+    @CommandArgument('match', metavar='MATCH', default=None, nargs='?',
+        help='Only display commands containing given substring.')
+    def debug_commands(self, match=None):
         import inspect
 
         handlers = self.context.commands.command_handlers
         for command in sorted(handlers.keys()):
+            if match and match not in command:
+                continue
+
             handler = handlers[command]
             cls = handler.cls
             method = getattr(cls, getattr(handler, 'method'))

python/mach/mach/commands/settings.py

@@ -2,11 +2,14 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this file,
 # You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import print_function, unicode_literals
+from __future__ import absolute_import, print_function, unicode_literals
 
 from textwrap import TextWrapper
 
-from mach.decorators import Command
+from mach.decorators import (
+    CommandProvider,
+    Command,
+)
 
 
 #@CommandProvider

python/mach/mach/config.py

@@ -25,7 +25,7 @@ msgfmt binary to perform this conversion. Generation of the original .po file
 can be done via the write_pot() of ConfigSettings.
 """
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import collections
 import gettext

python/mach/mach/decorators.py

@@ -2,22 +2,97 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import argparse
 import collections
 import inspect
 import types
 
-from .base import (
+from .base import MachError
-    MachError,
-    MethodHandler
-)
-
 from .config import ConfigProvider
 from .registrar import Registrar
 
 
+class _MachCommand(object):
+    """Container for mach command metadata.
+
+    Mach commands contain lots of attributes. This class exists to capture them
+    in a sane way so tuples, etc aren't used instead.
+    """
+    __slots__ = (
+        # Content from decorator arguments to define the command.
+        'name',
+        'subcommand',
+        'category',
+        'description',
+        'conditions',
+        '_parser',
+        'arguments',
+        'argument_group_names',
+
+        # Describes how dispatch is performed.
+
+        # The Python class providing the command. This is the class type not
+        # an instance of the class. Mach will instantiate a new instance of
+        # the class if the command is executed.
+        'cls',
+
+        # Whether the __init__ method of the class should receive a mach
+        # context instance. This should only affect the mach driver and how
+        # it instantiates classes.
+        'pass_context',
+
+        # The name of the method providing the command. In other words, this
+        # is the str name of the attribute on the class type corresponding to
+        # the name of the function.
+        'method',
+
+        # Dict of string to _MachCommand defining sub-commands for this
+        # command.
+        'subcommand_handlers',
+    )
+
+    def __init__(self, name=None, subcommand=None, category=None,
+                 description=None, conditions=None, parser=None):
+        self.name = name
+        self.subcommand = subcommand
+        self.category = category
+        self.description = description
+        self.conditions = conditions or []
+        self._parser = parser
+        self.arguments = []
+        self.argument_group_names = []
+
+        self.cls = None
+        self.pass_context = None
+        self.method = None
+        self.subcommand_handlers = {}
+
+    @property
+    def parser(self):
+        # Creating CLI parsers at command dispatch time can be expensive. Make
+        # it possible to lazy load them by using functions.
+        if callable(self._parser):
+            self._parser = self._parser()
+
+        return self._parser
+
+    @property
+    def docstring(self):
+        return self.cls.__dict__[self.method].__doc__
+
+    def __ior__(self, other):
+        if not isinstance(other, _MachCommand):
+            raise ValueError('can only operate on _MachCommand instances')
+
+        for a in self.__slots__:
+            if not getattr(self, a):
+                setattr(self, a, getattr(other, a))
+
+        return self
+
+
 def CommandProvider(cls):
     """Class decorator to denote that it provides subcommands for Mach.
 
@@ -47,6 +122,8 @@ def CommandProvider(cls):
         if len(spec.args) == 2:
             pass_context = True
 
+    seen_commands = set()
+
     # We scan __dict__ because we only care about the classes own attributes,
     # not inherited ones. If we did inherited attributes, we could potentially
     # define commands multiple times. We also sort keys so commands defined in
@@ -57,45 +134,80 @@ def CommandProvider(cls):
         if not isinstance(value, types.FunctionType):
             continue
 
-        command_name, category, description, conditions, parser = getattr(
+        command = getattr(value, '_mach_command', None)
-            value, '_mach_command', (None, None, None, None, None))
+        if not command:
-
-        if command_name is None:
             continue
 
-        if conditions is None and Registrar.require_conditions:
+        # Ignore subcommands for now: we handle them later.
+        if command.subcommand:
+            continue
+
+        seen_commands.add(command.name)
+
+        if not command.conditions and Registrar.require_conditions:
             continue
 
         msg = 'Mach command \'%s\' implemented incorrectly. ' + \
               'Conditions argument must take a list ' + \
               'of functions. Found %s instead.'
 
-        conditions = conditions or []
+        if not isinstance(command.conditions, collections.Iterable):
-        if not isinstance(conditions, collections.Iterable):
+            msg = msg % (command.name, type(command.conditions))
-            msg = msg % (command_name, type(conditions))
             raise MachError(msg)
 
-        for c in conditions:
+        for c in command.conditions:
             if not hasattr(c, '__call__'):
-                msg = msg % (command_name, type(c))
+                msg = msg % (command.name, type(c))
                 raise MachError(msg)
 
-        arguments = getattr(value, '_mach_command_args', None)
+        command.cls = cls
+        command.method = attr
+        command.pass_context = pass_context
 
-        argument_group_names = getattr(value, '_mach_command_arg_group_names', None)
+        Registrar.register_command_handler(command)
 
-        handler = MethodHandler(cls, attr, command_name, category=category,
+    # Now do another pass to get sub-commands. We do this in two passes so
-            description=description, conditions=conditions, parser=parser,
+    # we can check the parent command existence without having to hold
-            arguments=arguments, argument_group_names=argument_group_names,
+    # state and reconcile after traversal.
-            pass_context=pass_context)
+    for attr in sorted(cls.__dict__.keys()):
+        value = cls.__dict__[attr]
 
-        Registrar.register_command_handler(handler)
+        if not isinstance(value, types.FunctionType):
+            continue
+
+        command = getattr(value, '_mach_command', None)
+        if not command:
+            continue
+
+        # It is a regular command.
+        if not command.subcommand:
+            continue
+
+        if command.name not in seen_commands:
+            raise MachError('Command referenced by sub-command does not '
+                'exist: %s' % command.name)
+
+        if command.name not in Registrar.command_handlers:
+            continue
+
+        command.cls = cls
+        command.method = attr
+        command.pass_context = pass_context
+        parent = Registrar.command_handlers[command.name]
+
+        if parent._parser:
+            raise MachError('cannot declare sub commands against a command '
+                'that has a parser installed: %s' % command)
+        if command.subcommand in parent.subcommand_handlers:
+            raise MachError('sub-command already defined: %s' % command.subcommand)
+
+        parent.subcommand_handlers[command.subcommand] = command
 
     return cls
 
 
 class Command(object):
-    """Decorator for functions or methods that provide a mach subcommand.
+    """Decorator for functions or methods that provide a mach command.
 
     The decorator accepts arguments that define basic attributes of the
     command. The following arguments are recognized:
@@ -105,8 +217,9 @@ class Command(object):
 
          description -- A brief description of what the command does.
 
-         parser -- an optional argparse.ArgumentParser instance to use as
+         parser -- an optional argparse.ArgumentParser instance or callable
-             the basis for the command arguments.
+             that returns an argparse.ArgumentParser instance to use as the
+             basis for the command arguments.
 
     For example:
 
@@ -114,20 +227,45 @@ class Command(object):
         def foo(self):
             pass
     """
-    def __init__(self, name, category=None, description=None, conditions=None,
+    def __init__(self, name, **kwargs):
-                 parser=None):
+        self._mach_command = _MachCommand(name=name, **kwargs)
-        self._name = name
-        self._category = category
-        self._description = description
-        self._conditions = conditions
-        self._parser = parser
 
     def __call__(self, func):
-        func._mach_command = (self._name, self._category, self._description,
+        if not hasattr(func, '_mach_command'):
-                              self._conditions, self._parser)
+            func._mach_command = _MachCommand()
+
+        func._mach_command |= self._mach_command
 
         return func
 
+class SubCommand(object):
+    """Decorator for functions or methods that provide a sub-command.
+
+    Mach commands can have sub-commands. e.g. ``mach command foo`` or
+    ``mach command bar``. Each sub-command has its own parser and is
+    effectively its own mach command.
+
+    The decorator accepts arguments that define basic attributes of the
+    sub command:
+
+        command -- The string of the command this sub command should be
+        attached to.
+
+        subcommand -- The string name of the sub command to register.
+
+        description -- A textual description for this sub command.
+    """
+    def __init__(self, command, subcommand, description=None):
+        self._mach_command = _MachCommand(name=command, subcommand=subcommand,
+                                          description=description)
+
+    def __call__(self, func):
+        if not hasattr(func, '_mach_command'):
+            func._mach_command = _MachCommand()
+
+        func._mach_command |= self._mach_command
+
+        return func
 
 class CommandArgument(object):
     """Decorator for additional arguments to mach subcommands.
@@ -152,17 +290,16 @@ class CommandArgument(object):
         self._command_args = (args, kwargs)
 
     def __call__(self, func):
-        command_args = getattr(func, '_mach_command_args', [])
+        if not hasattr(func, '_mach_command'):
+            func._mach_command = _MachCommand()
 
-        command_args.insert(0, self._command_args)
+        func._mach_command.arguments.insert(0, self._command_args)
-
-        func._mach_command_args = command_args
 
         return func
 
 
 class CommandArgumentGroup(object):
-    """Decorator for additional argument groups to mach subcommands.
+    """Decorator for additional argument groups to mach commands.
 
     This decorator should be used to add arguments groups to mach commands.
     Arguments to the decorator are proxied to
@@ -185,11 +322,10 @@ class CommandArgumentGroup(object):
         self._group_name = group_name
 
     def __call__(self, func):
-        command_arg_group_names = getattr(func, '_mach_command_arg_group_names', [])
+        if not hasattr(func, '_mach_command'):
+            func._mach_command = _MachCommand()
 
-        command_arg_group_names.insert(0, self._group_name)
+        func._mach_command.argument_group_names.insert(0, self._group_name)
-
-        func._mach_command_arg_group_names = command_arg_group_names
 
         return func
 

python/mach/mach/dispatcher.py

@@ -2,7 +2,7 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import argparse
 import difflib
@@ -11,6 +11,7 @@ import sys
 from operator import itemgetter
 
 from .base import (
+    MachError,
     NoCommandError,
     UnknownCommandError,
     UnrecognizedArgumentError,
@@ -95,37 +96,71 @@ class CommandAction(argparse.Action):
             if command == 'help':
                 if args and args[0] not in ['-h', '--help']:
                     # Make sure args[0] is indeed a command.
-                    self._handle_subcommand_help(parser, args[0])
+                    self._handle_command_help(parser, args[0])
                 else:
                     self._handle_main_help(parser, namespace.verbose)
                 sys.exit(0)
             elif '-h' in args or '--help' in args:
                 # -h or --help is in the command arguments.
-                self._handle_subcommand_help(parser, command)
+                self._handle_command_help(parser, command)
                 sys.exit(0)
         else:
             raise NoCommandError()
 
         # Command suggestion
         if command not in self._mach_registrar.command_handlers:
+            # Make sure we don't suggest any deprecated commands.
+            names = [h.name for h in self._mach_registrar.command_handlers.values()
+                        if h.cls.__name__ == 'DeprecatedCommands']
             # We first try to look for a valid command that is very similar to the given command.
-            suggested_commands = difflib.get_close_matches(command, self._mach_registrar.command_handlers.keys(), cutoff=0.8)
+            suggested_commands = difflib.get_close_matches(command, names, cutoff=0.8)
             # If we find more than one matching command, or no command at all, we give command suggestions instead
             # (with a lower matching threshold). All commands that start with the given command (for instance: 'mochitest-plain',
             # 'mochitest-chrome', etc. for 'mochitest-') are also included.
             if len(suggested_commands) != 1:
-                suggested_commands = set(difflib.get_close_matches(command, self._mach_registrar.command_handlers.keys(), cutoff=0.5))
+                suggested_commands = set(difflib.get_close_matches(command, names, cutoff=0.5))
-                suggested_commands |= {cmd for cmd in self._mach_registrar.command_handlers if cmd.startswith(command)}
+                suggested_commands |= {cmd for cmd in names if cmd.startswith(command)}
                 raise UnknownCommandError(command, 'run', suggested_commands)
             sys.stderr.write("We're assuming the '%s' command is '%s' and we're executing it for you.\n\n" % (command, suggested_commands[0]))
             command = suggested_commands[0]
 
         handler = self._mach_registrar.command_handlers.get(command)
 
-        # FUTURE
+        usage = '%(prog)s [global arguments] ' + command + \
-        # If we wanted to conditionally enable commands based on whether
+            ' [command arguments]'
-        # it's possible to run them given the current state of system, here
+
-        # would be a good place to hook that up.
+        subcommand = None
+
+        # If there are sub-commands, parse the intent out immediately.
+        if handler.subcommand_handlers:
+            if not args:
+                self._handle_subcommand_main_help(parser, handler)
+                sys.exit(0)
+            elif len(args) == 1 and args[0] in ('help', '--help'):
+                self._handle_subcommand_main_help(parser, handler)
+                sys.exit(0)
+            # mach <command> help <subcommand>
+            elif len(args) == 2 and args[0] == 'help':
+                subcommand = args[1]
+                subhandler = handler.subcommand_handlers[subcommand]
+                self._handle_subcommand_help(parser, command, subcommand, subhandler)
+                sys.exit(0)
+            # We are running a sub command.
+            else:
+                subcommand = args[0]
+                if subcommand[0] == '-':
+                    raise MachError('%s invoked improperly. A sub-command name '
+                        'must be the first argument after the command name.' %
+                        command)
+
+                if subcommand not in handler.subcommand_handlers:
+                    raise UnknownCommandError(subcommand, 'run',
+                        handler.subcommand_handlers.keys())
+
+                handler = handler.subcommand_handlers[subcommand]
+                usage = '%(prog)s [global arguments] ' + command + ' ' + \
+                    subcommand + ' [command arguments]'
+                args.pop(0)
 
         # We create a new parser, populate it with the command's arguments,
         # then feed all remaining arguments to it, merging the results
@@ -134,12 +169,12 @@ class CommandAction(argparse.Action):
 
         parser_args = {
             'add_help': False,
-            'usage': '%(prog)s [global arguments] ' + command +
+            'usage': usage,
-                ' [command arguments]',
         }
 
         if handler.parser:
             subparser = handler.parser
+            subparser.context = self._context
         else:
             subparser = argparse.ArgumentParser(**parser_args)
 
@@ -166,6 +201,7 @@ class CommandAction(argparse.Action):
         # not interfere with arguments passed to the command.
         setattr(namespace, 'mach_handler', handler)
         setattr(namespace, 'command', command)
+        setattr(namespace, 'subcommand', subcommand)
 
         command_namespace, extra = subparser.parse_known_args(args)
         setattr(namespace, 'command_args', command_namespace)
@@ -251,12 +287,31 @@ class CommandAction(argparse.Action):
 
         parser.print_help()
 
-    def _handle_subcommand_help(self, parser, command):
+    def _populate_command_group(self, parser, handler, group):
+        extra_groups = {}
+        for group_name in handler.argument_group_names:
+            group_full_name = 'Command Arguments for ' + group_name
+            extra_groups[group_name] = \
+                parser.add_argument_group(group_full_name)
+
+        for arg in handler.arguments:
+            # Apply our group keyword.
+            group_name = arg[1].get('group')
+            if group_name:
+                del arg[1]['group']
+                group = extra_groups[group_name]
+            group.add_argument(*arg[0], **arg[1])
+
+    def _handle_command_help(self, parser, command):
         handler = self._mach_registrar.command_handlers.get(command)
 
         if not handler:
             raise UnknownCommandError(command, 'query')
 
+        if handler.subcommand_handlers:
+            self._handle_subcommand_main_help(parser, handler)
+            return
+
         # This code is worth explaining. Because we are doing funky things with
         # argument registration to allow the same option in both global and
         # command arguments, we can't simply put all arguments on the same
@@ -274,6 +329,7 @@ class CommandAction(argparse.Action):
 
         if handler.parser:
             c_parser = handler.parser
+            c_parser.context = self._context
             c_parser.formatter_class = NoUsageFormatter
             # Accessing _action_groups is a bit shady. We are highly dependent
             # on the argparse implementation not changing. We fail fast to
@@ -293,31 +349,84 @@ class CommandAction(argparse.Action):
             c_parser = argparse.ArgumentParser(**parser_args)
             group = c_parser.add_argument_group('Command Arguments')
 
-        extra_groups = {}
+        self._populate_command_group(c_parser, handler, group)
-        for group_name in handler.argument_group_names:
-            group_full_name = 'Command Arguments for ' + group_name
-            extra_groups[group_name] = \
-                c_parser.add_argument_group(group_full_name)
 
-        for arg in handler.arguments:
+        # Set the long help of the command to the docstring (if present) or
-            # Apply our group keyword.
+        # the command decorator description argument (if present).
-            group_name = arg[1].get('group')
+        if handler.docstring:
-            if group_name:
+            parser.description = format_docstring(handler.docstring)
-                del arg[1]['group']
+        elif handler.description:
-                group = extra_groups[group_name]
+            parser.description = handler.description
-            group.add_argument(*arg[0], **arg[1])
-
-        # This will print the description of the command below the usage.
-        description = handler.description
-        if description:
-            parser.description = description
 
         parser.usage = '%(prog)s [global arguments] ' + command + \
             ' [command arguments]'
+
+        # This is needed to preserve line endings in the description field,
+        # which may be populated from a docstring.
+        parser.formatter_class = argparse.RawDescriptionHelpFormatter
         parser.print_help()
         print('')
         c_parser.print_help()
 
+    def _handle_subcommand_main_help(self, parser, handler):
+        parser.usage = '%(prog)s [global arguments] ' + handler.name + \
+            ' subcommand [subcommand arguments]'
+        group = parser.add_argument_group('Sub Commands')
+
+        for subcommand, subhandler in sorted(handler.subcommand_handlers.iteritems()):
+            group.add_argument(subcommand, help=subhandler.description,
+                action='store_true')
+
+        if handler.docstring:
+            parser.description = format_docstring(handler.docstring)
+
+        parser.formatter_class = argparse.RawDescriptionHelpFormatter
+
+        parser.print_help()
+
+    def _handle_subcommand_help(self, parser, command, subcommand, handler):
+        parser.usage = '%(prog)s [global arguments] ' + command + \
+            ' ' + subcommand + ' [command arguments]'
+
+        c_parser = argparse.ArgumentParser(add_help=False,
+            formatter_class=CommandFormatter)
+        group = c_parser.add_argument_group('Sub Command Arguments')
+        self._populate_command_group(c_parser, handler, group)
+
+        if handler.docstring:
+            parser.description = format_docstring(handler.docstring)
+
+        parser.formatter_class = argparse.RawDescriptionHelpFormatter
+
+        parser.print_help()
+        print('')
+        c_parser.print_help()
+
+
 class NoUsageFormatter(argparse.HelpFormatter):
     def _format_usage(self, *args, **kwargs):
         return ""
+
+
+def format_docstring(docstring):
+    """Format a raw docstring into something suitable for presentation.
+
+    This function is based on the example function in PEP-0257.
+    """
+    if not docstring:
+        return ''
+    lines = docstring.expandtabs().splitlines()
+    indent = sys.maxint
+    for line in lines[1:]:
+        stripped = line.lstrip()
+        if stripped:
+            indent = min(indent, len(line) - len(stripped))
+    trimmed = [lines[0].strip()]
+    if indent < sys.maxint:
+        for line in lines[1:]:
+            trimmed.append(line[indent:].rstrip())
+    while trimmed and not trimmed[-1]:
+        trimmed.pop()
+    while trimmed and not trimmed[0]:
+        trimmed.pop(0)
+    return '\n'.join(trimmed)

python/mach/mach/main.py

@@ -25,7 +25,11 @@ from .base import (
     UnrecognizedArgumentError,
 )
 
-from .decorators import CommandProvider
+from .decorators import (
+    CommandArgument,
+    CommandProvider,
+    Command,
+)
 
 from .config import ConfigSettings
 from .dispatcher import CommandAction
@@ -87,13 +91,6 @@ It looks like you passed an unrecognized argument into mach.
 The %s command does not accept the arguments: %s
 '''.lstrip()
 
-INVALID_COMMAND_CONTEXT = r'''
-It looks like you tried to run a mach command from an invalid context. The %s
-command failed to meet the following conditions: %s
-
-Run |mach help| to show a list of all commands available to the current context.
-'''.lstrip()
-
 INVALID_ENTRY_POINT = r'''
 Entry points should return a list of command providers or directories
 containing command providers. The following entry point is invalid:
@@ -153,7 +150,7 @@ class ContextWrapper(object):
         except AttributeError as e:
             try:
                 ret = object.__getattribute__(self, '_handler')(self, key)
-            except AttributeError, TypeError:
+            except (AttributeError, TypeError):
                 # TypeError is in case the handler comes from old code not
                 # taking a key argument.
                 raise e
@@ -421,41 +418,17 @@ To see more help for a specific command, run:
             raise MachError('ArgumentParser result missing mach handler info.')
 
         handler = getattr(args, 'mach_handler')
-        cls = handler.cls
-
-        if handler.pass_context:
-            instance = cls(context)
-        else:
-            instance = cls()
-
-        if handler.conditions:
-            fail_conditions = []
-            for c in handler.conditions:
-                if not c(instance):
-                    fail_conditions.append(c)
-
-            if fail_conditions:
-                print(self._condition_failed_message(handler.name, fail_conditions))
-                return 1
-
-        fn = getattr(instance, handler.method)
 
         try:
-            result = fn(**vars(args.command_args))
+            return Registrar._run_command_handler(handler, context=context,
-
+                debug_command=args.debug_command, **vars(args.command_args))
-            if not result:
-                result = 0
-
-            assert isinstance(result, (int, long))
-
-            return result
         except KeyboardInterrupt as ki:
             raise ki
         except Exception as e:
             exc_type, exc_value, exc_tb = sys.exc_info()
 
-            # The first frame is us and is never used.
+            # The first two frames are us and are never used.
-            stack = traceback.extract_tb(exc_tb)[1:]
+            stack = traceback.extract_tb(exc_tb)[2:]
 
             # If we have nothing on the stack, the exception was raised as part
             # of calling the @Command method itself. This likely means a
@@ -502,16 +475,6 @@ To see more help for a specific command, run:
         self.logger.log(level, format_str,
             extra={'action': action, 'params': params})
 
-    @classmethod
-    def _condition_failed_message(cls, name, conditions):
-        msg = ['\n']
-        for c in conditions:
-            part = ['  %s' % c.__name__]
-            if c.__doc__ is not None:
-                part.append(c.__doc__)
-            msg.append(' - '.join(part))
-        return INVALID_COMMAND_CONTEXT % (name, '\n'.join(msg))
-
     def _print_error_header(self, argv, fh):
         fh.write('Error running mach:\n\n')
         fh.write('    ')
@@ -598,6 +561,8 @@ To see more help for a specific command, run:
         global_group.add_argument('-h', '--help', dest='help',
             action='store_true', default=False,
             help='Show this help message.')
+        global_group.add_argument('--debug-command', action='store_true',
+            help='Start a Python debugger when command is dispatched.')
 
         for args, kwargs in self.global_arguments:
             global_group.add_argument(*args, **kwargs)

python/mach/mach/registrar.py

@@ -2,10 +2,17 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 from .base import MachError
 
+INVALID_COMMAND_CONTEXT = r'''
+It looks like you tried to run a mach command from an invalid context. The %s
+command failed to meet the following conditions: %s
+
+Run |mach help| to show a list of all commands available to the current context.
+'''.lstrip()
+
 
 class MachRegistrar(object):
     """Container for mach command and config providers."""
@@ -38,15 +45,17 @@ class MachRegistrar(object):
         self.categories[name] = (title, description, priority)
         self.commands_by_category[name] = set()
 
-    def dispatch(self, name, context=None, **args):
+    @classmethod
-        """Dispatch/run a command.
+    def _condition_failed_message(cls, name, conditions):
+        msg = ['\n']
+        for c in conditions:
+            part = ['  %s' % c.__name__]
+            if c.__doc__ is not None:
+                part.append(c.__doc__)
+            msg.append(' - '.join(part))
+        return INVALID_COMMAND_CONTEXT % (name, '\n'.join(msg))
 
-        Commands can use this to call other commands.
+    def _run_command_handler(self, handler, context=None, debug_command=False, **kwargs):
-        """
-
-        # TODO The logic in this function overlaps with code in
-        # mach.main.Main._run() and should be consolidated.
-        handler = self.command_handlers[name]
         cls = handler.cls
 
         if handler.pass_context and not context:
@@ -57,9 +66,49 @@ class MachRegistrar(object):
         else:
             instance = cls()
 
+        if handler.conditions:
+            fail_conditions = []
+            for c in handler.conditions:
+                if not c(instance):
+                    fail_conditions.append(c)
+
+            if fail_conditions:
+                print(self._condition_failed_message(handler.name, fail_conditions))
+                return 1
+
         fn = getattr(instance, handler.method)
 
-        return fn(**args) or 0
+        if debug_command:
+            import pdb
+            result = pdb.runcall(fn, **kwargs)
+        else:
+            result = fn(**kwargs)
+
+        result = result or 0
+        assert isinstance(result, (int, long))
+        return result
+
+    def dispatch(self, name, context=None, argv=None, **kwargs):
+        """Dispatch/run a command.
+
+        Commands can use this to call other commands.
+        """
+        # TODO handler.subcommand_handlers are ignored
+        handler = self.command_handlers[name]
+
+        if handler.parser:
+            parser = handler.parser
+
+            # save and restore existing defaults so **kwargs don't persist across
+            # subsequent invocations of Registrar.dispatch()
+            old_defaults = parser._defaults.copy()
+            parser.set_defaults(**kwargs)
+            kwargs, _ = parser.parse_known_args(argv or [])
+            kwargs = vars(kwargs)
+            parser._defaults = old_defaults
+
+        return self._run_command_handler(handler, context=context, **kwargs)
+
 
 
 Registrar = MachRegistrar()

python/mach/mach/terminal.py

@@ -8,7 +8,7 @@ All the terminal interaction code is consolidated so the complexity can be in
 one place, away from code that is commonly looked at.
 """
 
-from __future__ import print_function, unicode_literals
+from __future__ import absolute_import, print_function, unicode_literals
 
 import logging
 import sys

python/mach/mach/test/common.py

@@ -9,6 +9,7 @@ import os
 import unittest
 
 from mach.main import Mach
+from mach.base import CommandContext
 
 here = os.path.abspath(os.path.dirname(__file__))
 

python/mach/mach/test/providers/throw.py

@@ -4,6 +4,8 @@
 
 from __future__ import unicode_literals
 
+import time
+
 from mach.decorators import (
     CommandArgument,
     CommandProvider,

python/mach/mach/test/test_conditions.py

@@ -8,6 +8,7 @@ import os
 
 from mach.base import MachError
 from mach.main import Mach
+from mach.registrar import Registrar
 from mach.test.common import TestBase
 
 from mozunit import main
@@ -48,14 +49,14 @@ class TestConditions(TestBase):
             result, stdout, stderr = self._run_mach([name])
             self.assertEquals(1, result)
 
-            fail_msg = Mach._condition_failed_message(name, fail_conditions)
+            fail_msg = Registrar._condition_failed_message(name, fail_conditions)
             self.assertEquals(fail_msg.rstrip(), stdout.rstrip())
 
         for name in ('cmd_bar_ctx', 'cmd_foobar_ctx'):
             result, stdout, stderr = self._run_mach([name], _populate_context)
             self.assertEquals(1, result)
 
-            fail_msg = Mach._condition_failed_message(name, fail_conditions)
+            fail_msg = Registrar._condition_failed_message(name, fail_conditions)
             self.assertEquals(fail_msg.rstrip(), stdout.rstrip())
 
     def test_invalid_type(self):

python/mach/mach/test/test_entry_point.py

@@ -11,6 +11,8 @@ from mach.base import MachError
 from mach.test.common import TestBase
 from mock import patch
 
+from mozunit import main
+
 
 here = os.path.abspath(os.path.dirname(__file__))