/sig instrumentation
/wg structured-logging

Hello @pohly

v1.24 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00pm PT on Thursday Feb 3rd, 2022. This enhancement is targeting alpha for v1.24,

Here’s where this enhancement currently stands:

• Updated KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for this release
• KEP has a test plan section filled out.
• KEP has up to date graduation criteria.
• KEP has a production readiness review that has been completed and merged into k/enhancements.

The status of this enhancement is marked as tracked. Please keep the issue description and the targeted stage up-to-date for release v1.24.
Thanks!

@encodeflush : the KEP PR was merged, all criteria for alpha in 1.24 should be met now.

Hi @pohly 👋 1.24 Docs shadow here.

This enhancement is marked as 'Needs Docs' for the 1.24 release.

Please follow the steps detailed in the documentation to open a PR against the dev-1.24 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thu March 31, 11:59 PM PDT.

Also, if needed take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thanks!

Hi @pohly 👋 1.24 Release Comms team here.

We have an opt-in process for the feature blog delivery. If you would like to publish a feature blog for this issue in this cycle, then please opt in on this tracking sheet.

The deadline for submissions and the feature blog freeze is scheduled for 01:00 UTC Wednesday 23rd March 2022 / 18:00 PDT Tuesday 22nd March 2022. Other important dates for delivery and review are listed here: https://github.com/kubernetes/sig-release/tree/master/releases/release-1.24#timeline.

For reference, here is the blog for 1.23.

Please feel free to reach out any time to me or on the #release-comms channel with questions or comments.

Thanks!

Hello @pohly

I'm just checking in once more as we approach the 1.24 Code Freeze on 18:00 PDT, Tuesday, March 29th 2022

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are merged by the code freeze deadline.
• Have a documentation placeholder PR open by 18:00 PDT, Thursday, March 31, 2022.
  • features: add ContextualLogging website#32452
  • logging: include information about contextual logging community#6560

For note, the status of this enhancement is currently marked as tracked.

Thank you!

/assign

I have added two doc PRs to the description.

/milestone clear

@pohly can we close this?

/assign @serathius

Hi @pohly Sry for the inconvenience. but I won't be able to follow these two PRs and have closed them, could you help with re-assign/release these two features?
staging/src/k8s.io/apimachinery (kubernetes/kubernetes#115317)
staging/src/k8s.io/kubectl (kubernetes/kubernetes#115087)

Please wait with submitting PRs. I need to more time to actually look at some of the packages before I can provide guidance on how to proceed.

Some design discussion of component cliant-go/rest migration to contextual logging

1. Usually the client-go returns the rest.Request and the rest.Result for the caller to use. The creation of their instances is controlled by the client-go. Can we define the logger as a struct field? like:

   type Request struct {
       ...
       logger *klog.Logger
       ...
   }
   
   func (r *Request) Fun() {
       r.log().Info(...)
   }
   
   // If the caller does not define it, a default one is returned
   func (r *Request) log() klog.Logger {
       if r.logger == nil {
           return klog.Background().WithName("rest_request")
       }
       return *r.logger
   }

2. The creation of the rest.Config is controlled by the caller, we add a context parameter for some functions used to build rest.Config, like:
   • Directly modify:

     func InClusterConfig(ctx context.Context) (*Config, error) {
         logger := klog.FromContext(ctx).WithName("rest_config")
         ...
     }

   • Consider compatibility:

     func InClusterConfig() (*Config, error) {
         return InClusterConfigWithContext(context.TODO())
     }
     
     func InClusterConfigWithContext(ctx context.Context) (*Config, error) {
         logger := klog.FromContext(ctx).WithName("rest_config")
         ...
     }

   Should we consider compatibility?
3. Can other structs used internally refer to question 1, and can we add a context parameter for some internal use functions?

Let me elaborate further... the problem with changing client-go or any other package under staging is that we cannot simply change an API. It breaks to much existing code. Instead, we have to extend the API in a backwards compatible way. Adding a klog.TODO or some other TODO remark doesn't help because it doesn't solve the API problem.

But our work isn't done at that point. Adding a new API is pointless if it doesn't get used by the Kubernetes components that we maintain. Out-of-tree components may also want to know that they should switch to the new API. Adding a "Deprecated" remark is too strong, the existing APIs are fine.

What I came up with is //logcheck:context as a special remark that tells logcheck to complain about an API, but only in code which cares about contextual logging. This isn't a solution in all cases, but at least when adding a WithContext variant it works.

So whoever now starts converting some package first has to look at existing usage of an API and then figure out how to change the API and that code - this is not easy, so beware before signing up to do this!

Thanks for your guidance, I combed through structs and functions call relationships in rest packages.

1. I add a logger field to the Request{} and Result{} structs and use methods to control their behavior:

   type Request struct {
       ...
       // logger you can set it using the SetLoggerFromContext(..) method.
       logger *logr.Logger
   }
   
   // SetLoggerWithContext retrieves a logger set by the caller and creates a new logger
   // with the constant name for the Reqeust's field logger.
   func (r *Request) SetLoggerFromContext(ctx context.Context) {
       inst := klog.FromContext(ctx).WithName(loggerNameRequest)
       r.logger = &inst
   }
   
   // log returns a not nil logger to be used by the methods. If the logger field is not defined,
   // sets a default logger for it.
   func (r *Request) log() logr.Logger {
       if r.logger == nil {
           def := klog.Background().WithName(loggerNameRequest)
           r.logger = &def
       }
       return *r.logger
   }

   Request public methods can use r.log() to write structured logging, and some private methods can use it to build contextual logging to call internal functions and tool methods.
2. They are internal functions and tool methods for the Request{} in urlbackoff.go, warnings.go and with_retry.go , so they can be converted to contextual logging.
3. These funcionts in plugin.go are called by AuthProvider implementations init(), so add some comment tags //logcheck:context to these functions.
4. Some of Config's creation functions are used infrequently, perhaps only once per project, so I figured callers wouldn't care much about the contextual logging, so add the commont tag //logcheck:context too.

I tried to revise my PR again, please check whether it is what you expected, thank you.

@WillardHu: This issue is not a good place to discuss API design aspects. Let's do that on Slack.

@WillardHu: This issue is not a good place to discuss API design aspects. Let's do that on Slack.

OK, thanks

/label lead-opted-in

Hello @pohly 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on Friday, February 9th, 2024 at 02:00 UTC.

This enhancement is targeting for stage beta for 1.30 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.30. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, we would just need to complete the following:

• Merge the KEP changes readme into the k/enhancements repo.
• Complete the PRR review process and merge it into k/enhancements.
• Mark this KEP as implementable for latest-milestone: 1.30.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

KEP PR for 1.30 got merged.

Hey @pohly - with all the requirements fulfilled this enhancement is now marked as tracked for the upcoming enhancements freeze 🚀! Thanks for your hard work

Hello 👋, 1.30 Enhancements team here.

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

I made an error, this question under scalability is now required in the KEP: https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template#can-enabling--using-this-feature-result-in-resource-exhaustion-of-some-node-resources-pids-sockets-inodes-etc

If you still wish to progress this enhancement in 1.30, please file an exception request. Thanks!

@tjons: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

/milestone clear

Hello @pohly 👋, 1.30 Docs Lead here.

Does this enhancement work planned for 1.30 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.30 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday February 22nd 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

Hi @pohly, @shivanshu1333, and @serathius,

👋 from the v1.30 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement!

We encourage blogs for features including, but not limited to: breaking changes, features and changes important to our users, and features that have been in progress for a long time and are graduating.

To opt in, you need to open a Feature Blog placeholder PR against the website repository.
The placeholder PR deadline is 27th February, 2024.

Here's the 1.30 Release Calendar

Doc PR for 1.30 created and linked to in the description

Hey again @pohly 👋 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 6th March 2024 .

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• promote contextual logging to beta, enabled by default kubernetes#122589
• slog support kubernetes#120696

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

Hi @pohly - checking in again here. There are about ~6 hours remaining until code freeze. Do you think you'll be able to merge kubernetes/kubernetes#120696 in time?

Hello @pohly 👋 , Enhancements team here.

With all the implementation(code related) PRs merged as per the issue description:

• promote contextual logging to beta, enabled by default kubernetes#122589

This enhancement is now marked as tracked for code freeze for the v1.30 Code Freeze!