Commit Graph

19686 Commits

Author SHA1 Message Date
chris meyers 7b087d4a6c loop over dnr nodes by topological sort
* Perform topological sort on graph nodes before looping over them to
mark do not run. This guarantees that parent nodes will be processed
before calling dependent child nodes. The complexity of the sorting is
N. The complexity of marking the the nodes is N*V
2018-11-27 16:12:41 -05:00
chris meyers cfa098479e Revert "optimize mark dnr nodes algorithm"
This reverts commit 6372c52772e26f64f3b4fd227ffb5e401d3688e1.
2018-11-27 16:12:41 -05:00
mabashian 3c510e6344 Fixed bug where root link became clickable. Fix workflow key on results page. 2018-11-27 16:12:41 -05:00
chris meyers 4c9a1d6b90 optimize mark dnr nodes algorithm
* Compute largest depth of each node and traverse graph by depth. This
allows us to check a node once, and only once, to determine if it needs
to be marked for do not run.
2018-11-27 16:12:41 -05:00
chris meyers d1aa52a2a6 fix up mark dnr logic 2018-11-27 16:12:41 -05:00
chris meyers f30f52a0a8 handle missing unified job template in workflow
* Workflow Node without unified_job_template is treated as a job marked
as failure; when deciding what path to execute.
* Remove optimization of marking dnr nodes due to it making the
algorithm incorrect.
2018-11-27 16:12:41 -05:00
mabashian 5b459e3c5d Code cleanup. Fixed bugs with workflow results page including details links 2018-11-27 16:12:41 -05:00
chris meyers 676c068b71 add job_description to failed workflow node
* When workflow job fails because a workflow job node doesn't have a
related unified_job_template note that with an error on the workflow
job's job_description
* When a workflow job fails because a failure path isn't defined, note
that on the workflow job job_description
2018-11-27 16:12:41 -05:00
chris meyers 00d71cea50 detect workflow nodes without job templates
* Fail workflow job run when encountering a Workflow Job Nodes with
no related job templates.
2018-11-27 16:12:41 -05:00
mabashian 72263c5c7b Addresses a number of workflow related bugs 2018-11-27 16:12:41 -05:00
chris meyers 281345dd67 flake8 fix 2018-11-27 16:12:41 -05:00
chris meyers 1a85fcd2d5 update docs to include workflow failure semantic 2018-11-27 16:12:41 -05:00
chris meyers c1171fe4ff treat canceled nodes as failed when processing wf
* When deciding what jobs to run next, treat canceled as failed.
* Also add tests.
2018-11-27 16:12:41 -05:00
chris meyers d6a8ad0b33 treat canceled jobs in wf the same as failed jobs
* Also fix spelling mistake that caused workflows to be falsely marked
successful in the case of a canceled job.
2018-11-27 16:12:41 -05:00
mabashian 4a6a3b27fa Fixed a number of workflow visualizer bugs. Added loading spinners while data is being loaded/processed. 2018-11-27 16:12:41 -05:00
chris meyers 266831e26d add cycle unit test 2018-11-27 16:12:41 -05:00
chris meyers a6e20eeaaa update wf done and failed tests 2018-11-27 16:12:41 -05:00
chris meyers 6529c1bb46 update done and fail detection for workflow
* Instead of traversing the workflow graph to determine if a workflow is
done or has failed; instead, loop through all the nodes in the graph and
grab only the relevant nodes.
2018-11-27 16:12:41 -05:00
mabashian ae0d0db62c Added dagre to handle our workflow graph layout. Fixed various workflow related bugs. 2018-11-27 16:12:41 -05:00
chris meyers b81d795c00 fix up dot graph generator
* Update graph dot generator to use the new efficient graph
2018-11-27 16:12:41 -05:00
chris meyers 1b87e11d8f flake8 2018-11-27 16:12:41 -05:00
chris meyers 8bb9cfd62a add dag tests 2018-11-27 16:12:41 -05:00
chris meyers a176a4b8cf remove unused code 2018-11-27 16:12:41 -05:00
chris meyers 3f4d14e48d crawl entire graph when marking DNR
* From the root, the code was only going down the did run path to find
nodes to mark DNR. This is incorrect, Now, we traverse the entire graph
each time to find nodes to mark DNR.
2018-11-27 16:12:41 -05:00
chris meyers 0499d419c3 more efficient graph processing
* Getting parent nodes from child was inefficient. Optimize it with a
hash table like we did for the getting of children.
* Getting leaf nodes was inefficient. Optimize it like we did getting
root nodes. A node is assumed to be a leaf node until it gets a child.
2018-11-27 16:12:41 -05:00
mabashian 700860e040 Fix long name tooltip. Fixed bug adding new node before finishing adding new link.
Fixed template list column layout.  Ensure that we're getting 200 workflow nodes per GET request
2018-11-27 16:12:41 -05:00
chris meyers 3dadeb3037 remove print statements 2018-11-27 16:12:41 -05:00
chris meyers 16a60412cf optimization fix
* WorkflowDAG accepts workflow job template and workflow jobs for which
to build a graph out of the nodes. The optimized query for each is
different. This changeset adds the differing queries for a workflow job.
2018-11-27 16:12:41 -05:00
chris meyers 9f3e272665 optimize cycle detection 2018-11-27 16:12:41 -05:00
mabashian b84fc3b111 Fixes for post-rebase bugs 2018-11-27 16:12:41 -05:00
chris meyers e1e8d3b372 bump migration 2018-11-27 16:12:40 -05:00
mabashian 05f4d94db2 Fixed serveral bugs including credential prompting. Added logic to bring links/nodes to the forefront when you hover over them in case there's some overlap 2018-11-27 16:12:40 -05:00
mabashian 61fb3eb390 First pass at implementing better node placement in the workflow graph 2018-11-27 16:12:40 -05:00
mabashian 7b95d2114d Implements workflow convergence without proper layout 2018-11-27 16:12:40 -05:00
chris meyers 07db7a41b3 more flake8 2018-11-27 16:12:40 -05:00
chris meyers 1120f8b1e1 try2 at the devil flake8 2018-11-27 16:12:40 -05:00
chris meyers 17b3996568 fix flake8 anyway I can 2018-11-27 16:12:40 -05:00
chris meyers 584b3f4e3d remove workflow test
* We now handle workflows with jobs that have errored. We treat them the
same as a failure result. Before, we would abort the workflow when we
encountered an error.
2018-11-27 16:12:40 -05:00
chris meyers f8c53f4933 handle job error state in convergence 2018-11-27 16:12:40 -05:00
chris meyers 6e40e9c856 handle edge case ring cycle 2018-11-27 16:12:40 -05:00
chris meyers 2f9dc4d075 remove relationship in view if cycle detected 2018-11-27 16:12:40 -05:00
chris meyers 9afc38b714 fixup migrations 2018-11-27 16:12:40 -05:00
chris meyers dfccc9e07d rework wf cycle detection for convergence 2018-11-27 16:12:40 -05:00
chris meyers 7b22d1b874 cycle detection when multiple parents 2018-11-27 16:12:40 -05:00
mabashian 29b4979736 Completed work necessary to support editing workflow links and nodes separately. Added hover and tooltip to links 2018-11-27 16:12:40 -05:00
mabashian 87d6253176 Decouple editing a wf node with editing a node link 2018-11-27 16:12:40 -05:00
chris meyers 1e10d4323f update docs 2018-11-27 16:12:40 -05:00
chris meyers 4111e53113 correctly name migration to align with 3.4.0 2018-11-27 16:12:40 -05:00
chris meyers 02df0c29e9 merge artifacts deterministically 2018-11-27 16:12:40 -05:00
chris meyers 475c90fd00 prevent job launching twice 2018-11-27 16:12:40 -05:00