Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Azure] Cross resource group nodes #604

Closed
feiskyer opened this issue Jul 31, 2018 · 43 comments
Closed

[Azure] Cross resource group nodes #604

feiskyer opened this issue Jul 31, 2018 · 43 comments
Assignees
@feiskyer
Labels
area/provider/azure Issues or PRs related to azure provider kind/feature Categorizes issue or PR as related to a new feature. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status
Projects
Provider Azure
Milestone
v1.16

Comments

@feiskyer
Copy link
Member

feiskyer commented Jul 31, 2018 •
edited by kacole2

Feature Description

  • One-line feature description (can be used as a release note): Add support of cross resource group nodes.
  • Primary contact (assignee): @feiskyer
  • Responsible SIGs: sig-azure
  • Design proposal link (community repo): Add proposal of cross resource group nodes for Azure community#2479
  • KEP: 20180809-cross-resource-group-nodes
  • Link to e2e and/or unit tests:
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @khenidak
  • Approver (likely from SIG/area to which feature belongs): @brendandburns
  • Feature target (which target equals to which milestone):
    • Alpha release target (1.12)
    • Beta release target (1.15)
    • Stable release target (x.y)
@feiskyer
Copy link
Member Author

feiskyer commented Jul 31, 2018 •
edited

/sig azure
/milestone v1.12

@k8s-ci-robot k8s-ci-robot added area/provider/azure Issues or PRs related to azure provider sig/storage Categorizes an issue or PR as relevant to SIG Storage. labels Jul 31, 2018
@feiskyer feiskyer removed the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Jul 31, 2018
@feiskyer
Copy link
Member Author

/kind feature
/stage alpha

@k8s-ci-robot k8s-ci-robot added stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status kind/feature Categorizes issue or PR as related to a new feature. labels Jul 31, 2018
@feiskyer feiskyer self-assigned this Jul 31, 2018
@feiskyer feiskyer added this to the v1.12 milestone Jul 31, 2018
@kacole2
Copy link
Contributor

kacole2 commented Jul 31, 2018

@feiskyer thank you. this has been added to the 1.12 Feature Tracking sheet. Please keep the post updated with links to PRs and design docs proposals.

@justaugustus justaugustus added the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Jul 31, 2018
@feiskyer
Copy link
Member Author

feiskyer commented Aug 6, 2018

Community PR created kubernetes/community#2479

@feiskyer feiskyer mentioned this issue Aug 8, 2018
Merged
@feiskyer feiskyer mentioned this issue Aug 20, 2018
Merged
@zparnold
Copy link
Member

Hey there! @feiskyer I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

@feiskyer
Copy link
Member Author

@zparnold We will update docs in repo https://github.com/kubernetes/cloud-provider-azure. Should it also be placeholded?

k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Aug 23, 2018
Merge pull request #67604 from feiskyer/cross-rg-nodes
82924e7 
Automatic merge from submit-queue (batch tested with PRs 66980, 67604, 67741, 67715). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support of Azure cross resource group nodes

**What this PR does / why we need it**:

Part of feature [Cross resource group nodes](kubernetes/enhancements#604).

This PR adds support of Azure cross resource group nodes that are labeled with `kubernetes.azure.com/resource-group=<rg-name>` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

See designs [here](kubernetes/community#2479).

**Release note**:

```release-note
Azure cloud provider now supports cross resource group nodes that are labeled with `kubernetes.azure.com/resource-group=<rg-name>` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`
```

/sig azure
/kind feature
@zparnold
Copy link
Member

zparnold commented Aug 25, 2018 via email

@justaugustus
Copy link
Member

@feiskyer -- it would be nice to have a blurb about configuring this on kubernetes.io

@feiskyer feiskyer mentioned this issue Aug 29, 2018
Merged
@feiskyer
Copy link
Member Author

@justaugustus Which folder do you suggest?

k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Aug 30, 2018
Merge pull request #67984 from feiskyer/on-prem
8aea674 
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add support for unmanaged nodes for Azure cloud provider

**What this PR does / why we need it**:

Continue of [Azure cross resource groups feature](kubernetes/enhancements#604).

This PR adds support for unmanaged nodes (such as on-prem or on other clouds) that are labeled with `alpha.service-controller.kubernetes.io/exclude-balancer=true` and `kubernetes.azure.com/managed=false`. Azure cloud provider would exclude such nodes from LoadBalancer backends and always assumes they are existing.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

See KEP [here](https://github.com/kubernetes/community/blob/master/keps/sig-azure/0025-20180809-cross-resource-group-nodes.md).

**Special notes for your reviewer**:

Azure cloud provider won't provision network routes for on-prem nodes, so cluster admins should ensure the network (including pod-to-pod, pod-to-node and node-to-node connectivity) has been set up properly.

**Release note**:

```release-note
Azure cloud provider now supports unmanaged nodes (such as on-prem) that are labeled with `kubernetes.azure.com/managed=false` and `alpha.service-controller.kubernetes.io/exclude-balancer=true`
```

/assign @khenidak @andyzhangx
/sig azure
/kind feature
/milestone v1.12
@feiskyer
Copy link
Member Author

feiskyer commented Sep 6, 2018

@justaugustus Yep, codes have been merged now and docs will be added to https://github.com/kubernetes/cloud-provider-azure

@feiskyer feiskyer mentioned this issue Sep 6, 2018
Merged
@justaugustus
Copy link
Member

Thanks for the update!

@ameukam
Copy link
Member

ameukam commented Oct 5, 2018

Hi folks,
Kubernetes 1.13 is going to be a 'stable' release since the cycle is only 10 weeks. We encourage no big alpha features and only consider adding this feature if you have a high level of confidence it will make code slush by 11/09. Are there plans for this enhancement to graduate to alpha/beta/stable within the 1.13 release cycle? If not, can you please remove it from the 1.12 milestone or add it to 1.13?

We are also now encouraging that every new enhancement aligns with a KEP. If a KEP has been created, please link to it in the original post. Please take the opportunity to develop a KEP.

@feiskyer
Copy link
Member Author

feiskyer commented Oct 5, 2018

It's already part of KEP, kubernetes/community#2479. Let's keep it in v1.13.

/milestone v1.13

@k8s-ci-robot k8s-ci-robot modified the milestones: v1.12, v1.13 Oct 5, 2018
@kacole2
Copy link
Contributor

kacole2 commented Oct 5, 2018

@feiskyer can you please update the original post with milestone of beta for 1.13? Thanks!
/stage beta

@kacole2 kacole2 added the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Apr 14, 2019
@justaugustus justaugustus added this to KEPs in Provider Azure Apr 28, 2019
@kacole2
Copy link
Contributor

kacole2 commented Apr 29, 2019

@feiskyer, we're doing a KEP review for enhancements to be included in the Kubernetes v1.15 milestone. After reviewing your KEP, it's currently missing test plans and graduation criteria which is required information per the KEP Template. Please update the KEP to include the required information before the Kubernetes 1.15 Enhancement Freeze date of 4/30/2019. Thank you

@kacole2
Copy link
Contributor

kacole2 commented May 1, 2019

@feiskyer, Enhancement Freeze for Kubernetes 1.15 has passed and this did not meet the deadline. Still missing Test Plans and Graduation Criteria from the KEP. This is now being removed from the 1.15 milestone and the tracking sheet. If there is a need for this to be in 1.15, please file an Enhancement Exception. Thank you. @claurence

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.15 milestone May 1, 2019
@kacole2 kacole2 added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels May 1, 2019
@feiskyer
Copy link
Member Author

feiskyer commented Jul 5, 2019

/milestone v1.16

@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Jul 5, 2019
@feiskyer feiskyer added stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team and removed stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team labels Jul 5, 2019
@feiskyer feiskyer mentioned this issue Jul 5, 2019
Closed
@rbitia
Copy link

rbitia commented Jul 11, 2019

Hi @feiskyer , I'm a 1.16 Enhancement Shadow. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If it's not graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thanks!

@feiskyer
Copy link
Member Author

@rbitia Yep, already added to 1.16 milestone

@rbitia
Copy link

rbitia commented Jul 15, 2019

Are you planning to graduate to stable or beta?

@justaugustus
Copy link
Member

Hey @rbitia! We're going GA/Stable with this in 1.16. :)

@feiskyer
Copy link
Member Author

Yep, we're working on the e2e tests for this.

@feiskyer feiskyer mentioned this issue Jul 31, 2019
Closed
@VineethReddy02
Copy link

Hi @feiskyer, I'm the v1.16 docs release shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.16) due by Friday,August 23rd. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

Thanks!

@feiskyer
Copy link
Member Author

feiskyer commented Aug 2, 2019

@VineethReddy02 Thanks, no new docs are required.

@kacole2
Copy link
Contributor

kacole2 commented Aug 5, 2019

@feiskyer urgent request can you please update the KEP with Graduation Criteria. This is a requirement for being included in the current 1.16 milestone. Please respond back with the PR ASAP. Thank you.

@feiskyer
Copy link
Member Author

feiskyer commented Aug 6, 2019

@kacole2 thanks, would do

@feiskyer
Copy link
Member Author

feiskyer commented Aug 6, 2019

Update: PR sent at #1202

@feiskyer
Copy link
Member Author

feiskyer commented Aug 9, 2019

Documents have been added at https://github.com/kubernetes/cloud-provider-azure/blob/master/docs/using-cross-resource-group-nodes.md.

E2e tests have been added at kubernetes-sigs/cloud-provider-azure#205.

Because aks-engine doesn't support to provision cross-resource-group nodes, the e2e tests couldn't be run periodically yet. But I think aks-engine should not be the blocker for this feature.

/close

@k8s-ci-robot
Copy link
Contributor

@feiskyer: Closing this issue.

In response to this:

Documents have been added at https://github.com/kubernetes/cloud-provider-azure/blob/master/docs/using-cross-resource-group-nodes.md.

E2e tests have been added at kubernetes-sigs/cloud-provider-azure#205.

Because aks-engine doesn't support to provision cross-resource-group nodes, the e2e tests couldn't be run periodically yet. But I think aks-engine should not be the blocker for this feature.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Provider Azure automation moved this from KEPs to Done Aug 9, 2019
@mrbobbytables mrbobbytables removed the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Oct 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@feiskyer feiskyer

Labels
area/provider/azure Issues or PRs related to azure provider kind/feature Categorizes issue or PR as related to a new feature. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status
Projects
Provider Azure
  
Done
Milestone
v1.16
Development

No branches or pull requests