Просмотр исходного кода

Cp fix CVE 2024 34156 (#2969)

* Support openshift in cluster prometheus with kube-rbac-proxy enabled (#2944)

* Add lock on file check and download so that partially downloaded file… (#2857)

* Add locker to azure storage integration file download to prevent accessing file that has not finished downloading

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

* fix lock duplication

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

---------

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* doc: use markdown rather than HTML for header (#2868)

Signed-off-by: DemoYeti <164791169+DemoYeti@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* OCI cloud costs integration (#2870)

* draft of oci cloud costs

Signed-off-by: nickcurie <ncurie@kubecost.com>

* remove outdated package

Signed-off-by: nickcurie <ncurie@kubecost.com>

* integration pass

Signed-off-by: nickcurie <ncurie@kubecost.com>

* pending changes

Signed-off-by: nickcurie <ncurie@kubecost.com>

* more pending changes

Signed-off-by: nickcurie <ncurie@kubecost.com>

* finalize cloud cost integration

Signed-off-by: nickcurie <ncurie@kubecost.com>

* remove unnecessary logs

Signed-off-by: nickcurie <ncurie@kubecost.com>

---------

Signed-off-by: nickcurie <ncurie@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* bump dependencies for CVE. (#2903)

Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Revert "bump dependencies for CVE. (#2903)" (#2908)

This reverts commit 75d0e249684ede3006ea2f1e3618ed9023d6d015.

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* handle aws generated tags (#2896)

* Add support for aws generated tags for s3 and athena integrations

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

* add consts

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>

---------

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* add connection statuses (#2917)

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Fixes /savings/orphanedResources end point failing with 500 internal server when its unable to read disk.description (#2920)

* fix /orphanedResources end point failing with 500 internal server when 1 particular orphaned disk description is unable to be read

Signed-off-by: Alan Rodrigues <alanr5691@yahoo.com>

* making it log.Errorf and also have similar error message as error

Signed-off-by: Alan Rodrigues <alanr5691@yahoo.com>

---------

Signed-off-by: Alan Rodrigues <alanr5691@yahoo.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* update MAINTAINERS (#2926)

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Updated efficiency calculations for consistency. (#2930)

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* fix release script, make relative repo for better testing in fork (#2936)

* fix release script, make relative repo for better testing in fork
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

* make this work on forks
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

---------

Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Co-authored-by: nik-kc <127428785+nik-kc@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Make logs more testable (#2940)

* make logs more testable
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

* add tests
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

* update test
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Support openshift in cluster prometheus with kube-rbac-proxy enabled

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comments

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Atm/fix windows (#2957)

* add tests

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
(cherry picked from commit ea0d25a821dd0b0866784d6f7df69ffc704e2fa7)

* bugfix

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
(cherry picked from commit fe77b4747dc27d50171c3fc5585cd0e4b12d3647)

* fix test case

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
(cherry picked from commit 0e505a12cf6d09b78c6c7035ea9288100f37a6b5)

* walk back gomod changes

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

---------

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Update OCI provider cost metadata (#2902)

* Update OCI provider cost metadata

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>

* Update OCI provider cost metadata

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>

* Update OCI provider cost metadata

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>

* Update OCI provider cost metadata

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>

---------

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Co-authored-by: Ajay Tripathy <ajay@kubecost.com>
Co-authored-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Bump the go_modules group across 1 directory with 2 updates (#2832)

Bumps the go_modules group with 2 updates in the /core directory: google.golang.org/protobuf and [golang.org/x/net](https://github.com/golang/net).

Updates `google.golang.org/protobuf` from 1.32.0 to 1.33.0

Updates `golang.org/x/net` from 0.21.0 to 0.23.0
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  dependency-group: go_modules
- dependency-name: golang.org/x/net
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Minor typos and corrections (#2856)

Signed-off-by: Matt Ray <github@mattray.dev>
Co-authored-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Minor typos (#2828)

* Minor typos

Signed-off-by: Matt Ray <github@mattray.dev>

* Update opencost-specv01.md

Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

---------

Signed-off-by: Matt Ray <github@mattray.dev>
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Co-authored-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* chore: add new region (#2949)

Signed-off-by: Ethonwu <ethonwu26@gmail.com>
Signed-off-by: ethon-wu <ethon_wu@trendmicro.com>
Co-authored-by: ethon-wu <ethon_wu@trendmicro.com>
Co-authored-by: Ajay Tripathy <ajay@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Implement plugin API upgrades (#2956)

* Implement plugin API upgrades

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* add tests

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* bugfix

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* fix test case

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* impl sorting

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* additional testing

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

* undo gomod changes

Signed-off-by: Alex Meijer <ameijer@kubecost.com>

---------

Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Support openshift in cluster prometheus with kube-rbac-proxy enabled

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comments

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Improve code

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* fix comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Support openshift in cluster prometheus with kube-rbac-proxy enabled

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comments

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Support openshift in cluster prometheus with kube-rbac-proxy enabled

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* address comments

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* Improve code

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

* fix comment

Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>

---------

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>
Signed-off-by: DemoYeti <164791169+DemoYeti@users.noreply.github.com>
Signed-off-by: nickcurie <ncurie@kubecost.com>
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Alan Rodrigues <alanr5691@yahoo.com>
Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Matt Ray <github@mattray.dev>
Signed-off-by: Ethonwu <ethonwu26@gmail.com>
Signed-off-by: ethon-wu <ethon_wu@trendmicro.com>
Co-authored-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: DemoYeti <164791169+DemoYeti@users.noreply.github.com>
Co-authored-by: Nick Curie <32180999+nickcurie@users.noreply.github.com>
Co-authored-by: Cliff Colvin <ccolvin@kubecost.com>
Co-authored-by: Alan Rodrigues <alanrodrigues@kubecost.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Co-authored-by: nik-kc <127428785+nik-kc@users.noreply.github.com>
Co-authored-by: Anders Swanson <91502735+anders-swanson@users.noreply.github.com>
Co-authored-by: Ajay Tripathy <ajay@kubecost.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Ray <github@mattray.dev>
Co-authored-by: Ethon Wu <ethonwu26@gmail.com>
Co-authored-by: ethon-wu <ethon_wu@trendmicro.com>

* update go.mod for golang cve (#2968)

Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>

---------

Signed-off-by: Sean Holcomb <seanholcomb@gmail.com>
Signed-off-by: Ishaan Mittal <ishaanmittal123@gmail.com>
Signed-off-by: DemoYeti <164791169+DemoYeti@users.noreply.github.com>
Signed-off-by: nickcurie <ncurie@kubecost.com>
Signed-off-by: Cliff Colvin <ccolvin@kubecost.com>
Signed-off-by: Alan Rodrigues <alanr5691@yahoo.com>
Signed-off-by: Alex Meijer <ameijer@kubecost.com>
Signed-off-by: Nik Willwerth <nwillwerth@kubecost.com>
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Matt Ray <github@mattray.dev>
Signed-off-by: Ethonwu <ethonwu26@gmail.com>
Signed-off-by: ethon-wu <ethon_wu@trendmicro.com>
Co-authored-by: Ishaan Mittal <ishaanmittal123@gmail.com>
Co-authored-by: Sean Holcomb <seanholcomb@gmail.com>
Co-authored-by: DemoYeti <164791169+DemoYeti@users.noreply.github.com>
Co-authored-by: Nick Curie <32180999+nickcurie@users.noreply.github.com>
Co-authored-by: Alan Rodrigues <alanrodrigues@kubecost.com>
Co-authored-by: Alex Meijer <ameijer@users.noreply.github.com>
Co-authored-by: nik-kc <127428785+nik-kc@users.noreply.github.com>
Co-authored-by: Anders Swanson <91502735+anders-swanson@users.noreply.github.com>
Co-authored-by: Ajay Tripathy <ajay@kubecost.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Ray <github@mattray.dev>
Co-authored-by: Ethon Wu <ethonwu26@gmail.com>
Co-authored-by: ethon-wu <ethon_wu@trendmicro.com>
Cliff Colvin 1 год назад
Родитель
Сommit
d2972cbc01
4 измененных файлов с 28 добавлено и 4 удалено
  1. 1 1
      core/go.mod
  2. 1 3
      go.mod
  3. 5 0
      pkg/env/costmodelenv.go
  4. 21 0
      pkg/prom/prom.go

+ 1 - 1
core/go.mod

@@ -1,6 +1,6 @@
 module github.com/opencost/opencost/core
 
-go 1.21.0
+go 1.22.7
 
 require (
 	github.com/davecgh/go-spew v1.1.1

+ 1 - 3
go.mod

@@ -195,6 +195,4 @@ require (
 	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
 )
 
-go 1.22.0
-
-toolchain go1.22.4
+go 1.22.7

+ 5 - 0
pkg/env/costmodelenv.go

@@ -72,6 +72,7 @@ const (
 	MultiClusterBearerToken       = "MC_BEARER_TOKEN"
 
 	InsecureSkipVerify = "INSECURE_SKIP_VERIFY"
+	KubeRbacProxyEnabled = "KUBE_RBAC_PROXY_ENABLED"
 
 	KubeConfigPathEnvVar = "KUBECONFIG_PATH"
 
@@ -382,6 +383,10 @@ func GetInsecureSkipVerify() bool {
 	return env.GetBool(InsecureSkipVerify, false)
 }
 
+func IsKubeRbacProxyEnabled() bool {
+	return env.GetBool(KubeRbacProxyEnabled, false)
+}
+
 // IsAggregateCostModelCacheDisabled returns the environment variable value for DisableAggregateCostModelCache which
 // will inform the aggregator on whether to load cached data. Defaults to false
 func IsAggregateCostModelCacheDisabled() bool {

+ 21 - 0
pkg/prom/prom.go

@@ -3,6 +3,7 @@ package prom
 import (
 	"context"
 	"crypto/tls"
+	"crypto/x509"
 	"fmt"
 	"net"
 	"net/http"
@@ -17,10 +18,13 @@ import (
 	"github.com/opencost/opencost/core/pkg/util/fileutil"
 	"github.com/opencost/opencost/core/pkg/util/httputil"
 	"github.com/opencost/opencost/core/pkg/version"
+	"github.com/opencost/opencost/pkg/env"
 
 	golog "log"
 
 	prometheus "github.com/prometheus/client_golang/api"
+	restclient "k8s.io/client-go/rest"
+	certutil "k8s.io/client-go/util/cert"
 )
 
 var UserAgent = fmt.Sprintf("Opencost/%s", version.Version)
@@ -374,6 +378,22 @@ type PrometheusClientConfig struct {
 
 // NewPrometheusClient creates a new rate limited client which limits by outbound concurrent requests.
 func NewPrometheusClient(address string, config *PrometheusClientConfig) (prometheus.Client, error) {
+
+	var tlsCaCert *x509.CertPool
+	// We will use the service account token and service-ca.crt to authenticate with the Prometheus server via kube-rbac-proxy.
+	// We need to ensure that the service account has the necessary permissions to access the Prometheus server by binding it to the appropriate role.
+	if env.IsKubeRbacProxyEnabled() {
+		restConfig, err := restclient.InClusterConfig()
+		if err != nil {
+			log.Errorf("KUBE_RBAC_PROXY_ENABLED was set to true but failed to get in-cluster config: %s", err)
+		}
+		config.Auth.BearerToken = restConfig.BearerToken
+		tlsCaCert, err = certutil.NewPool(`/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt`)
+		if err != nil {
+			log.Errorf("KUBE_RBAC_PROXY_ENABLED was set to true but failed to load service-ca.crt: %s", err)
+		}
+	}
+
 	// may be necessary for long prometheus queries
 	rt := httputil.NewUserAgentTransport(UserAgent, &http.Transport{
 		Proxy: http.ProxyFromEnvironment,
@@ -384,6 +404,7 @@ func NewPrometheusClient(address string, config *PrometheusClientConfig) (promet
 		TLSHandshakeTimeout: config.TLSHandshakeTimeout,
 		TLSClientConfig: &tls.Config{
 			InsecureSkipVerify: config.TLSInsecureSkipVerify,
+			RootCAs:            tlsCaCert,
 		},
 	})
 	pc := prometheus.Config{