Commit Graph

1177 Commits

Author SHA1 Message Date
Christian M. Adams
cea6d8c3cb Use utf-8 & properly parse hostname from registry URL 2021-05-25 14:44:42 -04:00
Christian M. Adams
8316a1d198 Create pull secret in cluster and use it in PodSpec
- base64 encode secret values before creating the secret
  - Construct valid .dockerconfigjson
  - Cancel jobs where it will obviously fail & error handling
  - Check if the secret exists first, then attempts to replace it if it does.
2021-05-25 14:44:42 -04:00
Christian M. Adams
a0840ddec2 Enforce EE pull policy on Container Groups 2021-05-25 14:44:41 -04:00
fedora
0c917caa2a --- Adding AWX_CONTAINER_GROUP_POD_LAUNCH_TIMEOUT in defaults.py
--- Passing this parameter in receptor_params in AWXReceptorJob class
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRIES from defaults.py as it was not being used anywhere
--- Removed AWX_CONTAINER_GROUP_POD_LAUNCH_RETRY_DELAY from defaults.py as it was not being used anywhere
2021-05-24 11:33:01 -04:00
softwarefactory-project-zuul[bot]
68f214c2be Merge pull request #8185 from AlanCoding/sniff_files_final
Use inventory and env private_data_dir subfolders

SUMMARY
This is another part of trying to get AWX to follow ansible-runner best practices.
See docs:
https://ansible-runner.readthedocs.io/en/stable/intro.html#runner-input-directory-hierarchy
So what runner & people expect is:
.
├── env
│   ├── envvars
│   ├── extravars
│   ├── passwords
│   ├── cmdline
│   ├── settings
│   └── ssh_key
├── inventory
│   └── hosts
└── project
    ├── test.yml
    └── roles
        └── testrole
            ├── defaults
            ├── handlers
            ├── meta
            ├── README.md
            ├── tasks
            ├── tests
            └── vars

What we were producing was something more like
├── cp
├── env
│   ├── cmdline
│   ├── envvars
│   ├── extravars
│   ├── passwords
│   ├── settings
│   └── ssh_key
├── project
│   ├── ansible.cfg
│   ├── ansible_env.yml
│   ├── async_tasks.yml
<snip>
│   └── vault.yml
└── tmp3evnorsu

There was no inventory folder, and inventory was stored in tmp3evnorsu as a top-level file. Any credential files were also saved in that same structure.
With this change, it's more like:
├── cp
├── env
│   ├── cmdline
│   ├── envvars
│   ├── extravars
│   ├── passwords
│   └── settings
├── inventory
│   └── tmp94xjm0d7
└── project
    ├── ansible.cfg
    ├── ansible_env.yml
    ├── async_tasks.yml
<snip>
    └── vault.yml

and any credential files (starting with tmp) are dumped into the env folder.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

API

AWX VERSION
14.1.0

ADDITIONAL INFORMATION
The motivation for this is that ansible-runner may require specific mounts for every file of directory in the private_data_dir. As such, we really don't want randomized filenames in the top level directory. I don't know for sure this is the direction we are going to go, but this seems like a defensible change on its own, to better adhere to the practices someone familiar with ansible-runner would expect.

Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Chris Meyers <None>
2021-05-19 17:24:33 +00:00
softwarefactory-project-zuul[bot]
282914e809 Merge pull request #10191 from chrismeyersfsu/fix-threading_tracebacks
close db and cache connection in new threads

It's not safe to share or re-use django db connections nor django cache connections across new threads nor processes.

Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Shane McDonald <me@shanemcd.com>
2021-05-18 19:41:58 +00:00
softwarefactory-project-zuul[bot]
fd60105db3 Merge pull request #10005 from AlanCoding/force_on_override
Update with force option if branch_override

Addresses a reported issue where force-pushing to feature branches did not work.
Connect #9371 (comment)

Reviewed-by: Bill Nottingham <None>
2021-05-17 23:58:31 +00:00
Sarabraj Singh
12cea1191e created a TransmitterThread class to start transmit jobs within. Surrounded ansible_runner.interface.run() in a try/except block to prevent the call from hanging on a socket pair that doesn't close 2021-05-17 14:35:13 -04:00
Alan Rominger
5dac46cd7d Update with force option if branch_override 2021-05-17 11:57:46 -04:00
Chris Meyers
9b7bc5109a close db and cache connection in new threads 2021-05-13 11:13:26 -04:00
Alan Rominger
ae320ab228 Do not set openstack env var to blank string 2021-05-13 08:57:19 -04:00
Alan Rominger
29c961e52a Remove comment 2021-05-13 08:57:19 -04:00
Alan Rominger
fd466c5cff Finish converting the runner strings 2021-05-13 08:57:19 -04:00
Alan Rominger
623cf0b4cd Start migrating the /runner path references 2021-05-13 08:57:18 -04:00
Alan Rominger
8f9373085a Fix credential env folder, test_tasks.py 2021-05-13 08:57:18 -04:00
Alan Rominger
11c5d577d6 Fix rel path for other inventories 2021-05-13 08:57:17 -04:00
Alan Rominger
0e17023ba3 Inventory directory already pre-created 2021-05-13 08:57:17 -04:00
Alan Rominger
f59da78328 Use inventory and env private_data_dir subfolders
This avoids writing files to the top level
  of the ansible-runner private_data_dir

Inventory is moved to be in the standard "inventory" folder

Credential related files are moved inside of the "env" folder

Also pre-create these folders when preparing for a job run

With this, args is the only top-level file still remaining
2021-05-13 08:56:38 -04:00
beeankha
fd5f3a82d2 Show error if no Execution Environment is found on project sync/job run 2021-05-12 11:52:12 -04:00
Yanis Guenane
562f78e53d Rename awx to automation for pod names 2021-05-04 14:17:45 +02:00
Shane McDonald
f94db36273 Forcibly ensure that the correct ansible-runner command is used for CG pods 2021-04-30 12:34:26 -04:00
Shane McDonald
77c573d64a Fix bug where incorrect EE was being used for Container Group pods 2021-04-30 12:21:03 -04:00
softwarefactory-project-zuul[bot]
6bea5dd294 Merge pull request #9957 from jbradberry/isolated-removal
Isolated removal

SUMMARY
Removal of the isolated nodes feature.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

API

AWX VERSION

Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Elyézer Rezende <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
2021-04-29 19:15:43 +00:00
Shane McDonald
8374b41e21 Improve error handling / display when Ansible Runner errors 2021-04-27 14:20:27 -04:00
Chris Meyers
afe4279e5f fix running analytics for the first time 2021-04-22 13:40:30 -04:00
Jeff Bradberry
6a599695db Remove the IsolatedManager and its associated playbooks and plugins 2021-04-22 10:17:02 -04:00
Alan Rominger
a54aab9717 Remove old uses of CustomInventoryScript
some uses are minor references

Fix some test fails
2021-04-22 08:36:48 -04:00
softwarefactory-project-zuul[bot]
2740155877 Merge pull request #9883 from shanemcd/remove-resource-profiling
Remove resource profiling feature

Reviewed-by: Ryan Petrello <ryan@ryanpetrello.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
2021-04-21 22:25:51 +00:00
Shane McDonald
21c493724c Remove resource profiling feature 2021-04-21 17:47:45 -04:00
Shane McDonald
658543c0fd Fix up some test fallout 2021-04-21 17:40:43 -04:00
Shane McDonald
17b8589ff2 Add option for ignoring tls on Container Registry credentials 2021-04-21 16:28:13 -04:00
Shane McDonald
bcf911daf1 Fix permission assignment on rendered registry auth files
- This file shouldnt need the executable bit
- Should have been setting permissions before writing any data
- No need to close the file since we're using open w/ a context manager
2021-04-21 16:03:11 -04:00
softwarefactory-project-zuul[bot]
eec4f8dcc2 Merge pull request #9973 from shanemcd/fix-adhoc-commands
Fix issue where ad-hoc commands for multiple hosts ran on single host

Should resolve #9685

Reviewed-by: Matthew Jones <bsdmatburt@gmail.com>
2021-04-21 17:17:36 +00:00
Shane McDonald
7fd5a4e79a Fix issue where ad-hoc commands for multiple hosts ran on single host 2021-04-21 12:41:51 -04:00
Seth Foster
8ab81216f3 unused import 2021-04-20 11:16:32 -04:00
Seth Foster
1e3cfdc986 just remove dangling images 2021-04-20 11:16:32 -04:00
Seth Foster
c64ec6bbf8 check if is_k8s 2021-04-20 11:16:32 -04:00
Seth Foster
4566e7a2a6 check subprocess returncode 2021-04-20 11:16:32 -04:00
Seth Foster
fa61ec6b3c Remove system job, replace with scheduled task 2021-04-20 11:16:31 -04:00
Shane McDonald
e30b4ca875 Dont reap pods when settings.RECEPTOR_RELEASE_WORK = False 2021-04-19 09:06:23 -04:00
softwarefactory-project-zuul[bot]
ad07d31b9a Merge pull request #9219 from mazhead/devel
Adding the scm_track_submodules option for project

SUMMARY
Adding the scm_track_submodules option which should fix the related #7846
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

API
UI

AWX VERSION
awx: 17.0.1

ADDITIONAL INFORMATION
This option will add the track_submodules option which is described in the ansible git module: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/git_module.html

if yes, submodules will track the latest commit on their master branch (or other branch specified in .gitmodules). If no, submodules will be kept at the revision specified by the main project. This is equivalent to specifying the --remote flag to git submodule update.

Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: None <None>
Reviewed-by: Alex Corey <Alex.swansboro@gmail.com>
2021-04-15 17:28:41 +00:00
Shane McDonald
4a62932ecd Store auth.json is pdd_wrapper directory 2021-04-13 11:54:43 -04:00
Rebeccah
e61d0c5cb7 credential validation for execution envs to allow only registry credentials to be associated with them, also adding security precautions for authfile and password, also combined token & password into one term to align with Quay, and added handling to account for users not filling in credential data and add a has_inputs function to simplify checking if the host, username, and password are present in the credential 2021-04-13 11:54:33 -04:00
Rebeccah
7c57aebd46 if there is a credential associated with an EE, create a JSON structure and write it to a file, then use that file to pull from protected registries from quay and edit the credential type for registries so that they combine the password and token fields into one field 2021-04-13 11:50:29 -04:00
Ryan Petrello
4db3c36ac3 record the receptor unit ID on the job record for debugging purposes
see: https://github.com/ansible/awx/issues/8641
2021-04-12 16:53:51 -04:00
mazhead
c6eb7da68d Adding the scm_track_submodules option which should fix the https://github.com/ansible/awx/issues/7846
Signed-off-by: mazhead <mazhead@gmail.com>
2021-04-12 16:22:23 +02:00
softwarefactory-project-zuul[bot]
868f68035f Merge pull request #9819 from shanemcd/fix-reaper
Update pod reaper to work with receptor launched pods

Reviewed-by: Ryan Petrello <None>
Reviewed-by: Alan Rominger <arominge@redhat.com>
2021-04-06 20:32:55 +00:00
Shane McDonald
2d48b24ef2 Update pod reaper to work with receptor launched pods 2021-04-05 17:45:15 -04:00
Shane McDonald
6294ddfded AWX_CONTAINER_GROUP_KEEP_POD -> RECEPTOR_KEEP_WORK 2021-04-05 17:44:40 -04:00
Jeff Bradberry
a448cb17d9 Refactor analytics.gather
With the change to use pk-based interval slicing for the job events
table, we need analytics.gather to be the code that manages all of the
"expensive" collector slicing.  While we are at it, let's ship each
chunked tarball file as we produce it.
2021-04-05 13:58:32 -04:00