* Each time a route is needed (i.e. when a task is sumitted to celery).
The router will be queried. This is ideal. With the previous method we
had to consider how a change in the routes would propogate to all celery
workers and nodes.
* fully describe the default awx queue
* Our dynamic queue registration would correct awx_private_queue.
However, we don't want celery to even create an "invalid"/extra
queue-exchange-route. This change makes sure we don't create extranious
things in rabbitmq.
* reduce the cluster queue registration output. Only output when the
queue registration list changes.
refactor existing handlers to be the related
"real" handler classes, which are swapped
out dynamically by external logger "proxy" handler class
real handler swapout only done on setting change
remove restart_local_services method
get rid of uWSGI fifo file
change TCP/UDP return type contract so that it mirrors
the request futures object
add details to socket error messages
* Using Kombu's default Broadcast() constructor requires only 1
parameter. That parameter defines the exchange name and the queue name
is randomly generated per-node.
* This caused problems if/when celery enters an infinite restart loop
because too many rabbit queues get created and rabbit OOM's
(gracefully).
* To remedy this we tell Broadcast the queue name to use, which is
derived from some constant + the node name so that the per-node queue
name is stable.
* Before, we had a special group, tower, that ran any async work that
tower needed done. This allowed users fine grain control over which
nodes did background work. However, this granularity was too complicated
for users. So now, all tower system work goes to a special non-user
exposed celery queue. Tower remains the fallback instance group to
execute jobs on. The tower group will be created upon install and
protected from deletion.
* Separates test messages from application messages
* Removes test runner and groups, processes, and streams from network_ui
* Adds network_ui_test
* Fixes routing for network_ui_test
* Removes coverage_report tool from network_ui
* Fixes network_ui_test test workflow
* Sets width and height of the page during tests
The ansible-network-ui prototype project builds a standalone Network UI
outside of Tower as its own Django application. The original prototype
code is located here:
https://github.com/benthomasson/ansible-network-ui.
The prototype provides a virtual canvas that supports placing
networking devices onto 2D plane and connecting those devices together
with connections called links. The point where the link connects
to the network device is called an interface. The devices, interfaces,
and links may all have their respective names. This models physical
networking devices is a simple fashion.
The prototype implements a pannable and zoomable 2D canvas in using SVG
elements and AngularJS directives. This is done by adding event
listeners for mouse and keyboard events to an SVG element that fills the
entire browser window.
Mouse and keyboard events are handled in a processing pipeline where
the processing units are implemented as finite state machines that
provide deterministic behavior to the UI.
The finite state machines are built in a visual way that makes
the states and transitions clearly evident. The visual tool for
building FSM is located here:
https://github.com/benthomasson/fsm-designer-svg. This tool
is a fork of this project where the canvas is the same. The elements
on the page are FSM states and the directional connections are called
transitions. The bootstrapping of the FSM designer tool and
network-ui happen in parallel. It was useful to try experiemental
code in FSM designer and then import it into network-ui.
The FSM designer tool provides a YAML description of the design
which can be used to generate skeleton code and check the implementation
against the design for discrepancies.
Events supported:
* Mouse click
* Mouse scroll-wheel
* Keyboard events
* Touch events
Interactions supported:
* Pan canvas by clicking-and-dragging on the background
* Zooming canvas by scrolling mousewheel
* Adding devices and links by using hotkeys
* Selecting devices, interaces, and links by clicking on their icon
* Editing labels on devices, interfaces, and links by double-clicking on
their icon
* Moving devices around the canvas by clicking-and-dragging on their
icon
Device types supported:
* router
* switch
* host
* racks
The database schema for the prototype is also developed with a visual
tool that makes the relationships in the snowflake schema for the models
quickly evident. This tool makes it very easy to build queries across
multiple tables using Django's query builder.
See: https://github.com/benthomasson/db-designer-svg
The client and the server communicate asynchronously over a websocket.
This allows the UI to be very responsive to user interaction since
the full request/response cycle is not needed for every user
interaction.
The server provides persistence of the UI state in the database
using event handlers for events generated in the UI. The UI
processes mouse and keyboard events, updates the UI, and
generates new types of events that are then sent to the server
to be persisted in the database.
UI elements are tracked by unique ids generated on the client
when an element is first created. This allows the elements to
be correctly tracked before they are stored in the database.
The history of the UI is stored in the TopologyHistory model
which is useful for tracking which client made which change
and is useful for implementing undo/redo.
Each message is given a unique id per client and has
a known message type. Message types are pre-populated
in the MessageType model using a database migration.
A History message containing all the change messages for a topology is
sent when the websocket is connected. This allows for undo/redo work
across sessions.
This prototype provides a server-side test runner for driving
tests in the user interface. Events are emitted on the server
to drive the UI. Test code coverage is measured using the
istanbul library which produces instrumented client code.
Code coverage for the server is is measured by the coverage library.
The test code coverage for the Python code is 100%.
* This also adds fields to the instance view for tracking cpu and
memory usage as well as information on what the capacity ranges are
* Also adds a flag for enabling/disabling instances which removes them
from all queues and has them stop processing new work
* The capacity is now based almost exclusively on some value relative
to forks
* capacity_adjustment allows you to commit an instance to a certain
amount of forks, cpu focused or memory focused
* Each job run adds a single fork overhead (that's the reasoning
behind the +1)
* Switch policy router queue to not be "tower" so that we don't
fall into a chicken/egg scenario
* Show fixed policy list in serializer so a user can determine if
an instance is manually managed
* Change IG membership mixin to not directly handle applying topology
changes. Instead it just makes sure the policy instance list is
accurate
* Add create/delete hooks for instances and groups to trigger policy
re-evaluation
* Update policy algorithm for fairer distribution
* Fix an issue where CELERY_ROUTES wasn't renamed after celery/django
upgrade
* Update unit tests to be more explicit
* Update count calculations used by algorithm to only consider
non-manual instances
* Adding unit tests and fixture
* Don't propagate logging messages from awx.main.tasks and
awx.main.scheduler
* Use advisory lock to prevent policy eval conflicts
* Allow updating instance groups from view
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process
add support for separate beat process
related to https://github.com/ansible/awx/issues/217
* Adds a configure tower in tower setting for users to configure a saml
attribute that tower will use to put users into teams and orgs.
