removing node exporter health check (#3004)

* Introducing OpenCost Guru on Gurubase.io (#2974)

Signed-off-by: Kursat Aktas <kursat.ce@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Dedupe logs. This function is continuously called multiple times each (#2982)

minute.

Signed-off-by: thomasvn <thomasnguyen96@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* welcome cliff! (#2980)

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Bump github.com/golang-jwt/jwt/v4 in the go_modules group (#2985)

Bumps the go_modules group with 1 update: [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt).

Updates `github.com/golang-jwt/jwt/v4` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.0...v4.5.1)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* ENG-3056: Stop double counting GPURequestAvg (#2987)

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Increate timeout for ratecardapi to 5 minutes (#2990)

Timeout of 1 minute is not enough to complete the ratecardQuery

Signed-off-by: Duncan Bakker <62943186+duncan485@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Fix: fetching for azure nodes that are not spot instances   (#2978)

* fix issue with non spot nodes fetching

Signed-off-by: Lars Barmettler <lrb@open.ch>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* restore unintented changes

Signed-off-by: Lars Barmettler <lrb@open.ch>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* fix repeated fetch for non spot instances

Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* fix PR changes

Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* Update pkg/cloud/azure/provider.go

Co-authored-by: Sean Holcomb <sean@kubecost.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* Update pkg/cloud/azure/provider.go

Co-authored-by: Sean Holcomb <sean@kubecost.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* Update pkg/cloud/azure/provider.go

Co-authored-by: Sean Holcomb <sean@kubecost.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* Update pkg/cloud/azure/provider.go

Co-authored-by: Sean Holcomb <sean@kubecost.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* refactor

Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

* fix bug

Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>

---------

Signed-off-by: Lars Barmettler <lrb@open.ch>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Co-authored-by: Lars Barmettler <lrb@open.ch>
Co-authored-by: Sean Holcomb <sean@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Revert "reduce memory consumption (#2725)" (#2998)

This reverts commit a3beb32763c5e56f7aafbb957724f1cdd8b197f0.

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* ENG-2956: Minor fix to only log if GPUUsageMax is not nil and isNaN (#2995)

* ENG-3056: Stop double counting GPURequestAvg

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>

* ENG-2956: Log only if GPUUsageMax is not nil and isNaN

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>

* ENG-2956: Update UT

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>

* ENG-2956: Update UT

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>

---------

Signed-off-by: Pranav Bhat <pbhat@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* removing network exporter health check

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* add comment for more clarity

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* remove the metrics instead of commenting it

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* add Thomas E to emeritus maintainer list (#3012)

Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* add support for adding local to the response objects (#3013)

Signed-off-by: fix test <>

Signed-off-by: fix test <>
Signed-off-by: fix test <ameijer@kubecost.com>
Co-authored-by: fix test <>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Refactor and add testing for Athena and S3 row parsing (#2916)

* Refactor and add testing for Athena and S3 row parsing

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

* Add extra test cases

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

* Replace const with string literals to make "quality" gate pass

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

---------

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Kubernetes cache optimization (#2736)

* delete cluster importer/exporter

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced pod and namespace structs

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced node struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced service struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced daemonset struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced deployment struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced statefulset struct

Signed-off-by: r2k1 <yokree@gmail.com>

* delete unused ReplicaSets ReplicationControllers watches

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced PersistentVolume struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced PersistentVolumeClaim struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced StorageClass struct

Signed-off-by: r2k1 <yokree@gmail.com>

* use reduced Job struct

Signed-off-by: r2k1 <yokree@gmail.com>

* delete unused PodDisruptionBudget

Signed-off-by: r2k1 <yokree@gmail.com>

* create reflector-based version of cluster cache

Signed-off-by: r2k1 <yokree@gmail.com>

* update uid getter

Signed-off-by: r2k1 <yokree@gmail.com>

* upgrade kube api version

Signed-off-by: r2k1 <yokree@gmail.com>

* rename NewKubernetesClusterCache2 => NewKubernetesClusterCacheV2

Signed-off-by: r2k1 <yokree@gmail.com>

* improve type safety

Signed-off-by: r2k1 <yokree@gmail.com>

* add replicationControllerStore, replicaSetStore, pdbStore

Signed-off-by: r2k1 <yokree@gmail.com>

* return cluster importer/exporter

Signed-off-by: r2k1 <yokree@gmail.com>

* group statements

Signed-off-by: r2k1 <yokree@gmail.com>

* fix pdb store reference

Signed-off-by: r2k1 <yokree@gmail.com>

* fix post-merge conflicts

Signed-off-by: r2k1 <yokree@gmail.com>

* Cluster Cache Optimization Effort - Additional Fields (#3008)

* merge develop

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

* First iteration on adding necessary fields to flyweights.

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

---------

Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: r2k1 <yokree@gmail.com>
Co-authored-by: r2k1 <yokree@gmail.com>

* make cache selection configurable

Signed-off-by: r2k1 <yokree@gmail.com>

* fix compilation error

Signed-off-by: r2k1 <yokree@gmail.com>

* Cluster Cache Optimization Part 2 - Config Map Separation (#3011)

* [Cache Optimization] Remove the k8s resource endpoints (#3016)

* Remove the k8s resource endpoints

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

* drop pod logs from opencost

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

---------

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

---------

Signed-off-by: r2k1 <yokree@gmail.com>
Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Co-authored-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Bump golang.org/x/crypto from 0.24.0 to 0.31.0 in the go_modules group (#3019)

Bumps the go_modules group with 1 update: [golang.org/x/crypto](https://github.com/golang/crypto).

Updates `golang.org/x/crypto` from 0.24.0 to 0.31.0
- [Commits](https://github.com/golang/crypto/compare/v0.24.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Fix Window Parsing in JSON (#3021)

Signed-off-by: fix test <ameijer@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* [Cluster Cache] Block to Pre-Initialize Lists (#3023)

* add blocking behavior on Run() to pre-initialize resource lists.

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

* Ensure we don't request resources for specific option.

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

---------

Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* remove node exporter metrics Id

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Remove legacy filters among other dead code paths (#2878)

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* GPU Label Pricing (#3024)

* [Cluster Cache] Block to Pre-Initialize Lists (#3023)

* add blocking behavior on Run() to pre-initialize resource lists.

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

* Ensure we don't request resources for specific option.

Signed-off-by: Matt Bolt <mbolt35@gmail.com>

---------

Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Added GPU label pricing fixes.

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Updated a unit test.

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Addressed PR feedback.

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Added unit test.

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Remove legacy filters among other dead code paths (#2878)

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

* Empty commit.

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>

---------

Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: Matt Bolt <mbolt35@gmail.com>
Co-authored-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

---------

Signed-off-by: Kursat Aktas <kursat.ce@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>
Signed-off-by: thomasvn <thomasnguyen96@gmail.com>
Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Pranav Bhat <pbhat@kubecost.com>
Signed-off-by: Duncan Bakker <62943186+duncan485@users.noreply.github.com>
Signed-off-by: Lars Barmettler <lrb@open.ch>
Signed-off-by: Lars Barmettler <barmettler.lars@gmail.com>
Signed-off-by: theMaimu <barmettler.lars@gmail.com>
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: fix test <>
Signed-off-by: fix test <ameijer@kubecost.com>
Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: r2k1 <yokree@gmail.com>
Signed-off-by: Matt Bolt <mbolt35@gmail.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Co-authored-by: Kursat Aktas <kursat.ce@gmail.com>
Co-authored-by: Thomas Nguyen <thomasvn.dev@gmail.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kubecost-pb <pbhat@kubecost.com>
Co-authored-by: Duncan Bakker <62943186+duncan485@users.noreply.github.com>
Co-authored-by: theMaimu <barmettler.lars@gmail.com>
Co-authored-by: Lars Barmettler <lrb@open.ch>
Co-authored-by: Sean Holcomb <sean@kubecost.com>
Co-authored-by: Cliff Colvin <ccolvin@kubecost.com>
Co-authored-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: r2k1 <yokree@gmail.com>
Co-authored-by: Matt Bolt <mbolt35@gmail.com>
Co-authored-by: nik-kc <127428785+nik-kc@users.noreply.github.com>

pkg/prom/diagnostics.go

@@ -26,9 +26,6 @@ const (
 	// KubecostDiagnosticMetricID is the identifier for the metric used to determine if Kubecost metrics are being scraped.
 	KubecostDiagnosticMetricID = "kubecostMetric"
 
-	// NodeExporterDiagnosticMetricID is the identifier for the metric used to determine if NodeExporter metrics are being scraped.
-	NodeExporterDiagnosticMetricID = "neMetric"
-
 	// ScrapeIntervalDiagnosticMetricID is the identifier for the metric used to determine if prometheus has its own self-scraped
 	// metrics.
 	ScrapeIntervalDiagnosticMetricID = "scrapeInterval"
@@ -78,13 +75,6 @@ var diagnosticDefinitions map[string]*diagnosticDefinition = map[string]*diagnos
 		Label:       "Kubecost metrics available",
 		Description: "Determine if metrics from Kubecost are available during last 5 minutes.",
 	},
-	NodeExporterDiagnosticMetricID: {
-		ID:          NodeExporterDiagnosticMetricID,
-		QueryFmt:    `absent_over_time(node_cpu_seconds_total{%s}[5m] %s)`,
-		Label:       "Node-exporter metrics available",
-		Description: "Determine if metrics from node-exporter are available during last 5 minutes.",
-		DocLink:     fmt.Sprintf("%s#node-exporter-metrics-available", DocumentationBaseURL),
-	},
 	CAdvisorLabelDiagnosticMetricID: {
 		ID:          CAdvisorLabelDiagnosticMetricID,
 		QueryFmt:    `absent_over_time(container_cpu_usage_seconds_total{container!="",pod!="", %s}[5m] %s)`,