Add sysctl support #34

sttts · 2016-07-20T09:05:40Z

Description

This feature aims at extending the current pod specification with support
for namespaced kernel parameters (sysctls) set for each pod.

Scope of work planned for v1.11

graduate to beta

Progress Tracker

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: IN_DEVELOPMENT

More advice:

Design

Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

Use as many PRs as you need. Write tests in the same or different PRs, as is convenient for you.
As each PR is merged, add a comment to this issue referencing the PRs. Code goes in the http://github.com/kubernetes/kubernetes repository,
and sometimes http://github.com/kubernetes/contrib, or other repos.
When you are done with the code, apply the "code-complete" label.
When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
testing. They won't do detailed code review: that already happened when your PRs were reviewed.
When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

Write user docs and get them merged in.
User docs go into http://github.com/kubernetes/kubernetes.github.io.
When the feature has user docs, please add a comment mentioning @kubernetes/docs.
When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

Automatic merge from submit-queue Add sysctl support Implementation of proposal #26057, feature kubernetes/enhancements#34 TODO: - [x] change types.go - [x] implement docker and rkt support - [x] add e2e tests - [x] decide whether we want apiserver validation - ~~[ ] add documentation~~: api docs exist. Existing PodSecurityContext docs is very light and links back to the api docs anyway: https://github.com/kubernetes/kubernetes.github.io/blob/6684555ed9e3121388d30e9c49ac6556ef0241e0/docs/user-guide/security-context.md - [x] change PodSecurityPolicy in types.go - [x] write admission controller support for PodSecurityPolicy - [x] write e2e test for PodSecurityPolicy - [x] make sure we are compatible in the sense of https://github.com/kubernetes/kubernetes/blob/master/docs/devel/api_changes.md - [x] test e2e with rkt: it only works with kubenet, not with no-op network plugin. The later has no sysctl support. - ~~[ ] add RunC implementation~~ (~~if that is already in kube,~~ it isn't) - [x] update whitelist - [x] switch PSC fields to annotations - [x] switch PSP fields to annotations - [x] decide about `--experimental-whitelist-sysctl` flag to be additive or absolute - [x] decide whether to add a sysctl node whitelist annotation ### Release notes: ```release-note The pod annotation `security.alpha.kubernetes.io/sysctls` now allows customization of namespaced and well isolated kernel parameters (sysctls), starting with `kernel.shm_rmid_forced`, `net.ipv4.ip_local_port_range`, `net.ipv4.tcp_max_syn_backlog` and `net.ipv4.tcp_syncookies` for Kubernetes 1.4. The pod annotation `security.alpha.kubernetes.io/unsafeSysctls` allows customization of namespaced sysctls where isolation is unclear. Unsafe sysctls must be enabled at-your-own-risk on the kubelet with the `--experimental-allowed-unsafe-sysctls` flag. Future versions will improve on resource isolation and more sysctls will be considered safe. ```

sttts · 2016-09-01T14:42:03Z

@kubernetes/docs here are the sysctl docs: kubernetes/website#1126

sttts · 2016-09-01T14:43:46Z

/cc @kubernetes/feature-reviewers

fejta-bot · 2018-01-02T14:00:39Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

sttts · 2018-03-09T19:28:14Z

There are a number of people using sysctls now. I have not heard any issues with them.

I suggest to promote the current API (transformed to native fields in the PSP and on pods) to beta for 1.11.

@jeremyeder @vishh @derekwaynecarr @php-coder

@kubernetes/sig-node-api-reviews

jeremyeder · 2018-03-21T11:48:51Z

Thanks, @sttts!

derekwaynecarr · 2018-03-21T19:43:06Z

@sttts it needs a feature gate.

from node side, it would be @sjenning who could help push this in sig-node. will sync w/ @dchen1107 next week. we discussed this briefly in last weeks sig-node.

sttts · 2018-03-21T19:50:07Z

@derekwaynecarr in the kubelet not much would change code-wise. But of course we need a "go" from the node team that they think using sysctls is safe enough for beta. Note, that graduation to beta does not say anything about extending the list of safe sysctls.

~~It's already feature gated. As beta we would switch the default to true.~~ Doesn't look like we had a feature gate sjenning/kubernetes@f4f7220

justaugustus · 2018-04-17T02:10:49Z

@sttts
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

Description
Milestone
Assignee(s)
Labels:
- stage/{alpha,beta,stable}
- sig/*
- kind/feature

cc @idvoretskyi

php-coder · 2018-04-17T07:47:05Z

@sttts Do we need to wait until pod annotations become fields or it doesn't block us from graduating it to beta?

liggitt · 2018-04-17T14:32:22Z

@sttts Do we need to wait until pod annotations become fields or it doesn't block us from graduating it to beta?

yes, they need to become fields

justaugustus · 2018-04-17T20:39:02Z

@php-coder @liggitt so just to clarify, no work planned for 1.11?
Also, would you mind updating the description to fit the new feature description template?

sttts · 2018-04-17T20:42:37Z

@justaugustus promotion to beta is discussed in sig-node /cc @derekwaynecarr

sjenning · 2018-04-23T20:55:26Z

/remove-lifecycle stale

pacoxu · 2021-02-08T06:28:39Z

We may update it.
/remove-lifecycle rotten

Graduation Criteria:

API changes allowing to configure the pod-scoped sysctl via spec.securityContext field.（cancelled）
Promote --experimental-allowed-unsafe-sysctls kubelet flag to kubelet config api option
feature gate enabled by default
e2e tests promote WIP
documentation

Some discussion in
https://docs.google.com/document/d/1FbThdQQVNPISNjK4IEqfliuRCA6pLbUiayb3OASOXHA/edit?usp=sharing

arunmk · 2021-02-08T06:48:00Z

Hi @pacoxu @ehashman

Enhancements Freeze is 2 days away, Feb 9th EOD PST

Enhancements team is aware that KEP update is currently in progress (as per comment). Please make sure to work on PRR questionnaires and requirements and get it merged before the freeze. For PRR related questions or to boost the PR for PRR review, please reach out in slack #prod-readiness

The KEP looks good.

Any enhancements that do not complete the following requirements by the freeze will require an exception.

[DONE] ~~The KEP must be merged in an implementable state: state is currently provisional~~
[DONE] ~~The KEP must have test plans~~
[DONE] ~~The KEP must have graduation criteria~~
[DONE] ~~The KEP must have a production readiness review: needs file under https://github.com/kubernetes/enhancements/tree/master/keps/prod-readiness/sig-node~~

EDIT: updated status in place. Thanks for the update @ehashman .

ehashman · 2021-02-08T19:32:46Z

I will try to get the KEP doc updated today.

ehashman · 2021-02-09T18:19:16Z

@arunmk this is now good to go for 1.21

arunmk · 2021-02-09T21:13:33Z

Thanks @ehashman . I am looking at it now and will update the status in-place.

arunmk · 2021-02-20T03:39:17Z

Hi @ehashman, @pacoxu,

Since your Enhancement is scheduled to be in 1.21, please keep in mind the important upcoming dates:

Tuesday, March 9th: Week 9 - Code Freeze
Tuesday, March 16th: Week 10 - Docs Placeholder PR deadline
- If this enhancement requires new docs or modification to existing docs, please follow the steps in the Open a placeholder PR doc to open a PR against k/website repo.

As a reminder, please link all of your k/k PR(s) and k/website PR(s) to this issue so we can track them.

Thanks!

pacoxu · 2021-02-20T03:55:01Z

I will work on the flag promotion today:

- Promote --experimental-allowed-unsafe-sysctls kubelet flag to kubelet config api option Done in 1.11(remove experimental) & 1.16(move to kubeadm/kubelet config)
WIP by @wgahnagl Graduate sysctls to GA kubernetes#99158 or 【Placeholder】make sure everything done and Promote sysctls to GA version kubernetes#99263 focus on lock feature gate on. (Will be done in 1.21 need to rebase and test fix)
WIP: Promote e2e to conformance testing Promote sysctls e2e test to Conformance kubernetes#99734
Check documentations: https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/

Read the history implementation and do a summary for this feature:

1.4 add security.alpha.kubernetes.io/unsafe-sysctls annotation support, move to client-go annotation_key_constants.go in 1.7 , move to pkg/api/ annotation_key_constants.go in 1.8, move to pkg/apis/core/ in 1.9-1.10.
1.11 Promote sysctl annotations to fields kubernetes#63717 Promote sysctl annotations to fields. Add feature gate sysctls, meanwhile, the kubelet "experimental-allowed-unsafe-sysctls" promote to "allowed-unsafe-sysctls".
1.14 Remove Sysctls feature gate from validation kubernetes#72752 Moves feature gate checking of Sysctls out of validation into strategy utility methods, and avoids dropping data on update.
1.16 kubelet: add allowed sysctl to KubeletConfiguration kubernetes#72974 by @sjenning： kubelet: add allowed sysctl to KubeletConfiguration (add support in kubeadm as well)
1.21 GA and lock to true: Graduate sysctls to GA kubernetes#99158
1.23 Remove the feature gate.

arunmk · 2021-03-02T19:41:49Z

Hi @pacoxu,

Enhancements team is currently tracking the following PR

Graduate sysctls to GA kubernetes#99158

With the PR merged, can we mark this enhancement complete for code freeze or do you have other PR(s) that are being worked on as part of the release?

Thanks

ehashman · 2021-03-02T22:30:30Z

@arunmk currently @wgahnagl is checking if we need to promote e2e tests to conformance as a result of the GA. There are no other code changes.

I'll see about the documentation changes for GA. @pacoxu do you want to take that on or should I find someone else?

pacoxu · 2021-03-03T00:55:16Z

@ehashman
I will update sysctls docs next week if no one is working on it.

arunmk · 2021-03-06T07:24:42Z

Hi @ehashman, @wgahnagl,

Could you mention if there is going to be a PR for the e2e tests? Code freeze is on 3/9 and it should make it by then. If it's not going to come in this KEP can be marked done.

Thanks!

arunmk · 2021-03-06T19:10:45Z

(Adding this as a note sent to all)

Hi @ehashman @wgahnagl,

A friendly reminder that Code freeze is 3 days away, March 9th EOD PST

Any enhancements that are NOT code complete by the freeze will be removed from the milestone and will require an exception to be added back.

Please also keep in mind that if this enhancement requires new docs or modification to existing docs, you'll need to follow the steps in the Open a placeholder PR doc to open a PR against k/website repo by March 16th EOD PST

Thanks!

ehashman · 2021-03-08T18:45:06Z

Hi @arunmk,

There is a PR for the e2es, however those aren't required to merge by code freeze, they will follow the test freeze deadline. Code changes are complete.

kubernetes/kubernetes#99734

tengqm · 2021-03-10T01:00:40Z

@ehashman @pacoxu Please open a placeholder PR in k/website for tracking. Thanks.

pacoxu · 2021-03-10T05:24:30Z

@tengqm I opened kubernetes/website#26981 for tracking.

annajung · 2021-04-08T14:25:56Z

Hi @ehashman, @pacoxu

Can you update the kep.yaml to reflect a status of implemented:

enhancements/keps/sig-node/34-sysctl-fields/kep.yaml

Line 19 in 29a891b

status: implementable

Once that merges, we can close out this issue.

pacoxu · 2021-04-08T15:17:12Z

Fine

…ubernetes#34) * Add feature page for following feature: Istio Authorizatio [Beta] * Update authorization.md Add link to blog: Large Scale Security Policy Performance Tests

This was referenced Jul 20, 2016

Add sysctl proposal kubernetes/kubernetes#26057

Closed

Add sysctl support kubernetes/kubernetes#27180

Merged

idvoretskyi added this to the v1.4 milestone Jul 20, 2016

idvoretskyi added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Aug 4, 2016

idvoretskyi assigned sttts Aug 4, 2016

goltermann added the alpha-in-1.4 label Aug 22, 2016

sttts mentioned this issue Aug 30, 2016

Add docs/admin/sysctls.md kubernetes/website#1126

Closed

idvoretskyi removed the (deprecated label - do not use) alpha-in-1.4 label Aug 8, 2017

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 2, 2018

sttts added the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Jan 2, 2018

k8s-ci-robot added the kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API label Mar 9, 2018

sttts modified the milestones: v1.4, v1.11 Mar 21, 2018

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 23, 2018

derekwaynecarr assigned derekwaynecarr and ingvagabund Apr 23, 2018

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Feb 8, 2021

ehashman mentioned this issue Feb 9, 2021

Migrate KEP-34 to new template #2471

Merged

ehashman mentioned this issue Feb 17, 2021

Graduate sysctls to GA kubernetes/kubernetes#99158

Merged

pacoxu mentioned this issue Feb 22, 2021

【Placeholder】make sure everything done and Promote sysctls to GA version kubernetes/kubernetes#99263

Closed

4 tasks

pacoxu mentioned this issue Mar 4, 2021

Promote sysctls e2e test to Conformance kubernetes/kubernetes#99734

Merged

pacoxu mentioned this issue Mar 11, 2021

Promote Sysctl to ga kubernetes/website#26981

Merged

ehashman mentioned this issue Apr 8, 2021

Mark sysctls as implemented #2613

Merged

JamesLaverack added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels Apr 25, 2021

k8s-ci-robot closed this as completed in #2613 May 11, 2021

This was referenced Aug 4, 2021

remove GAed feature gates sysctls kubernetes/kubernetes#104113

Merged

nominate pacoxu as sig node reviewer kubernetes/kubernetes#104186

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sysctl support #34

Add sysctl support #34

sttts commented Jul 20, 2016 •

edited

sttts commented Sep 1, 2016

sttts commented Sep 1, 2016

fejta-bot commented Jan 2, 2018

sttts commented Mar 9, 2018

jeremyeder commented Mar 21, 2018

derekwaynecarr commented Mar 21, 2018

sttts commented Mar 21, 2018 •

edited

justaugustus commented Apr 17, 2018

php-coder commented Apr 17, 2018

liggitt commented Apr 17, 2018

justaugustus commented Apr 17, 2018

sttts commented Apr 17, 2018

sjenning commented Apr 23, 2018

pacoxu commented Feb 8, 2021 •

edited

arunmk commented Feb 8, 2021 •

edited

ehashman commented Feb 8, 2021

ehashman commented Feb 9, 2021

arunmk commented Feb 9, 2021

arunmk commented Feb 20, 2021 •

edited

pacoxu commented Feb 20, 2021 •

edited

arunmk commented Mar 2, 2021 •

edited

ehashman commented Mar 2, 2021

pacoxu commented Mar 3, 2021

arunmk commented Mar 6, 2021

arunmk commented Mar 6, 2021

ehashman commented Mar 8, 2021

tengqm commented Mar 10, 2021

pacoxu commented Mar 10, 2021

annajung commented Apr 8, 2021

pacoxu commented Apr 8, 2021

Add sysctl support #34

Add sysctl support #34

Comments

sttts commented Jul 20, 2016 • edited

Description

Scope of work planned for v1.11

Progress Tracker

sttts commented Sep 1, 2016

sttts commented Sep 1, 2016

fejta-bot commented Jan 2, 2018

sttts commented Mar 9, 2018

jeremyeder commented Mar 21, 2018

derekwaynecarr commented Mar 21, 2018

sttts commented Mar 21, 2018 • edited

justaugustus commented Apr 17, 2018

php-coder commented Apr 17, 2018

liggitt commented Apr 17, 2018

justaugustus commented Apr 17, 2018

sttts commented Apr 17, 2018

sjenning commented Apr 23, 2018

pacoxu commented Feb 8, 2021 • edited

arunmk commented Feb 8, 2021 • edited

ehashman commented Feb 8, 2021

ehashman commented Feb 9, 2021

arunmk commented Feb 9, 2021

arunmk commented Feb 20, 2021 • edited

pacoxu commented Feb 20, 2021 • edited

arunmk commented Mar 2, 2021 • edited

ehashman commented Mar 2, 2021

pacoxu commented Mar 3, 2021

arunmk commented Mar 6, 2021

arunmk commented Mar 6, 2021

ehashman commented Mar 8, 2021

tengqm commented Mar 10, 2021

pacoxu commented Mar 10, 2021

annajung commented Apr 8, 2021

pacoxu commented Apr 8, 2021

sttts commented Jul 20, 2016 •

edited

sttts commented Mar 21, 2018 •

edited

pacoxu commented Feb 8, 2021 •

edited

arunmk commented Feb 8, 2021 •

edited

arunmk commented Feb 20, 2021 •

edited

pacoxu commented Feb 20, 2021 •

edited

arunmk commented Mar 2, 2021 •

edited