Commit Graph

31 Commits

Author SHA1 Message Date
Jeff Bradberry
a448cb17d9 Refactor analytics.gather
With the change to use pk-based interval slicing for the job events
table, we need analytics.gather to be the code that manages all of the
"expensive" collector slicing.  While we are at it, let's ship each
chunked tarball file as we produce it.
2021-04-05 13:58:32 -04:00
Ryan Petrello
c2ef0a6500 move code linting to a stricter pep8-esque auto-formatting tool, black 2021-03-23 09:39:58 -04:00
Alan Rominger
44e6e9344b Remove mercurial support 2020-12-01 14:06:34 -05:00
Ryan Petrello
4445d096f5 reimplement licensing to work with RHSM and entitlement uploads
Co-authored-by: Christian Adams <chadams@redhat.com>
2020-10-30 16:39:57 -04:00
Bill Nottingham
13802fcf2b Don't return error messages for license errors
Just log the exception and return None.
2020-09-10 21:19:07 -04:00
Bill Nottingham
c753324872 Move back to less frequent collections, and split large event tables
This should ensure we stay under 100MB at all times.
2020-09-09 17:42:40 -04:00
Bill Nottingham
9f67b6742c Fail more gracefully if analytics.ship() is called with a bad path,
or it's deleted out from under us.
2020-09-09 17:42:40 -04:00
Bill Nottingham
1a15f18be3 Stop using the TowerAnalyticsState solo model
This is now tracked in the AUTOMATION_ANALYTICS_LAST_GATHER setting.
2020-09-09 17:42:40 -04:00
Bill Nottingham
40309e6f70 Ensure we do not send large bundles, or empty bundles
Collect expensive collectors separately, and in a loop
where we make smaller intermediate dumps.

Don't return a table dump if there are no records, and
don't put that CSV in the manifest.

Fix up unit tests.
2020-09-09 17:42:40 -04:00
Bill Nottingham
1c4b06fe1e Refactor analytics collectors.
- Only have one registration class
- Add description fields
- Add automation collector information to /api/v2/config
2020-09-09 17:10:14 -04:00
Christian M. Adams
4ea648307e Accept all responses <300 from Insights API 2020-08-28 12:45:50 -04:00
Gabe Muniz
80234c5600 rework the tar to always delete the leftover directories 2020-03-16 19:54:15 +00:00
Bill Nottingham
71ef7cdec1 Use AWX_TASK_ENV when connecting to Red Hat services 2020-02-07 14:29:42 -05:00
Christian Adams
05e6f4ab3c Add User-Agent to analytics upload POST 2019-11-25 22:22:05 -05:00
Graham Mainwaring
7dd241fcff Add a --dry-run option to gather analytics locally, even if analytics is disabled in settings. 2019-10-17 13:54:13 -04:00
Christian Adams
9d81b00772 have analytics collections verify with system trusted CA list 2019-10-16 13:32:06 -04:00
Christian Adams
844b8a803f update frequency of collection for automation analytics 2019-10-09 21:10:48 -04:00
Christian Adams
6309c0a426 Upload using RH cred settings 2019-09-11 13:24:09 -04:00
Ben Thomasson
10d53637ad Changes uploader to use the insights api directly 2019-09-06 10:58:35 -04:00
Christian Adams
78c0d531bc Adds versions to analytics collectors and manifest file.
- adds 'query_info.json' to contain collection metadata
  - adds 'manifest.json' to contain collection file version info
2019-08-27 10:14:14 -04:00
Christian Adams
e72b2fac6d Delete collection tarball when no longer needed
* Delete after shipping it
   * Delete when ship() fails
2019-07-01 11:11:44 -04:00
softwarefactory-project-zuul[bot]
bd50e5d6a8 Merge pull request #3896 from rooftopcellist/collection_no_reset
collect events based on last ship, not collection

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-05-17 15:32:48 +00:00
Christian Adams
0c4925afe8 rm extra collection error line 2019-05-16 14:48:41 -04:00
Christian Adams
d0e160a037 collect events based on last ship, not collection 2019-05-14 17:14:44 -04:00
Christian Adams
add8673d7c rename analytics setting 2019-04-17 15:28:15 -04:00
Christian Adams
4a0778a3d5 exclude implicit project updates 2019-03-28 16:12:30 -04:00
Christian Adams
c481919a49 check for open license 2019-03-27 19:54:25 -04:00
Christian Adams
58f0e42bd6 update setting wording & add instance-enabled field 2019-03-27 19:54:25 -04:00
Christian Adams
a7368cec43 add copy events table and remove events queries
add copy unifiedjob table, futher optimization, rm since from job_instance queries
2019-03-27 19:53:00 -04:00
Christian Adams
636153d92c add insights setting, optimize and consolidate queries 2019-03-27 19:53:00 -04:00
Ryan Petrello
c586fa9821 add a minimal framework for generating analytics/metrics
annotate queries & add license analytics
2019-03-27 19:53:00 -04:00