Automatic merge from submit-queue

Make controller Run methods consistent

- startup/shutdown logging
- wait for cache sync logging
- defer utilruntime.HandleCrash()
- wait for stop channel before exiting

Please review for accuracy and consistency.

cc @deads2k @wojtek-t @smarterclayton @sttts @mikedanese @liggitt @rrati @derekwaynecarr

Automatic merge from submit-queue

Removed un-necessary empty line.

- startup/shutdown logging
- wait for cache sync logging
- defer utilruntime.HandleCrash()
- wait for stop channel before exiting

Automatic merge from submit-queue

Fix resource gatherer

Automatic merge from submit-queue (batch tested with PRs 44414, 44318)

Finish migration to edge-based for endpoints in KubeProxy

Ref #43702

Automatic merge from submit-queue (batch tested with PRs 44414, 44318)

Cluster autoscaler broken nodes E2E test fix

cc: @MaciekPytel @fgrzadkowski

Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)

use sudo in mv ssh cmd

**What this PR does / why we need it**:
Fixes _HostCleanup_ kubelet e2e test where `sudo` was missing from a ssh `mv` command. 
The exact test is:
`host cleanup with volume mounts [Volume][HostCleanup][Flaky] Host cleanup after disrupting NFS volume [NFS] move NFS client pod's UID directory then delete pod` This test has been consistently red in the gce-flaky suite.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)

Delete EmptyDir volume directly instead of renaming the directory.

**What this PR does / why we need it**:
The volume operation executor can handle duplicate requests on the same volume now, so it is not necessary to rename the directory anymore.  This change can cause pod deletion to take longer for large emptydir volumes because now the pod waits for the volume to be deleted until it continues pod cleanup.  But this is actually required for local disk scheduling so that we don't schedule new pods that need emptydir volumes on the node if the previous emptydir has not be fully reclaimed yet.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43534

**Special notes for your reviewer**:

**Release note**:

NONE

cc @kubernetes/sig-storage-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)

fix error message in ReplicaCalculator

**What this PR does / why we need it**: fixes spelling in an error message

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**: have previously signed the CLA for minikube, not sure if that covers this repo also.

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)

Drop leading path of KUBECTL.EXE if it shows up in User-Agent.

Partial fix for #44419 

Release note: kube-apiserver now drops unneeded path information if an older version of Windows kubectl sends it.

Automatic merge from submit-queue

Fix to not allow empty topology key when the feature AffinityInAnnotations is disabled.

Fix https://github.com/kubernetes/kubernetes/issues/44360
@davidopp @kubernetes/sig-scheduling-pr-reviews

Automatic merge from submit-queue

Missing --project in the cluster up scripts for gce

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)

Extend the gc admission plugin to check ownerReference.blockOwnerDeletion

#Extend the gc admission plugin to prevent user who doesn't have delete permission of the *owner* from changing blockOwnerDeletion field of existing ownerReferences, or adding ownerReference with blockOwnerDeletion=true.

The plugin need a RESTMapper to translate ownerRef.Kind to Resource. It should be using a dynamic one. However, as discussed in https://github.com/kubernetes/kubernetes/pull/42615, such a RESTMapper will be built after watchable discovery API is implemented, so in this PR the plugin is using the `api.Registry.RESTMapper()`, which is also [used](https://github.com/kubernetes/kubernetes/blob/master/cmd/kube-controller-manager/app/core.go#L165-L166) by the garbage collector currently.

```release-note
Extending the gc admission plugin so that a user who doesn't have delete permission of the *owner* cannot modify blockOwnerDeletion field of existing ownerReferences, or add new ownerReference with blockOwnerDeletion=true
```

cc @lavalamp

Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)

Show node address type in kubectl describe node command

**What this PR does / why we need it**:
To make node addresses more clear when describing node.

Before:
```
Addresses:		192.168.1.220,192.168.1.220,192.168.1.220
```

After:
```
Addresses:
 LegacyHostIP:	192.168.1.220
 InternalIP:	192.168.1.220
 Hostname:	192.168.1.220
```

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```

Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)

Move pkg/api/ref.go and pkg/api/resource_helpers.go to subpackages

First two commits are #44296 #44299. (unfortunately, removing these commits results in conflicts)

This PR moves resource_helpers.go to a subpackage. It's mostly a mechanic move, except that:
* i kept the methods of ResourceName and ResourceList in pkg/api/method.go

I'll send one more PR to separate api.Scheme etc. to their own package.

Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)

Update batch types.go docs

@bgrant0607 you asked about it in https://github.com/kubernetes/kubernetes/pull/42663#discussion_r109065758

Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)

Make resource gatherer get data about etcd resource usage in kubemark…

… setup

Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)

remove defaulting from conversion path

follow up for #42764

* remove call to defaulting from conversion path (defaulting is a separate step from conversion)
* remove non-top-level-object defaulting registration (unused after conversion call is removed)
* generate missing top-level defaults for some api groups:
  * autoscaling/v2alpha1
  * policy/v1alpha1
  * policy/v1beta1
* register top-level defaults for some api groups that were missing them:
  * autoscaling/v2alpha1
  * settings/v1alpha1

Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)

`failure-domain` is no longer used in command line arg.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)

ServiceAccountsController does not need nameIndexFunc to index ST

The ServiceAccountsController's Informer does not need nameIndexFunc.

Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)

Preserve int data when unmarshalling for TPR

**What this PR does / why we need it**:

The Go json package converts all numbers to float64 while unmarshalling.
This exposes many of the int64 fields to corruption when marshalled back to json.

The json package provided by kubernetes also provides a way to defer conversion of numbers
(https://golang.org/pkg/encoding/json/#Decoder.UseNumber) and does the conversions to int or float.

This is also implemented in the custom json package. See:
(https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apimachinery/pkg/util/json/json.go)

Now, the number is preserved as an integer till the highest int64 number - `9223372036854775807`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #30213

**Special notes for your reviewer**: See also https://github.com/kubernetes/kubernetes/pull/16964

**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)

Try in cluster config when input KubeConfig is empty

**What this PR does / why we need it**:
Allows for downstream providers to run e2es "incluster" sans kubeconfig

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
NONE
```

/cc @kubernetes/sig-testing-pr-reviews @marun

Automatic merge from submit-queue

Segregate storage e2es for easier assignment and dep isolation.

**What this PR does / why we need it**:
Follow on from slack conversations to isolate storage deps and have easier assignment of e2es.

As of today there is general consensus that the e2es need to be broken apart, this is 1st iteration on the sig-storage test splitting.  Follow on work is needed here.   

**Release note**:

```
NONE
```

/cc @saad-ali @kubernetes/sig-storage-pr-reviews @kubernetes/sig-testing-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Move pkg/api/ref.go to a subpackage

First commit is #44296. (unfortunately, removing that commit results in conflicts)

This PR moves the pkg/api/ref.go to its own subpackage. It's mostly a mechanic move.

I'll send a few more PRs to make the k8s.io/kubernetes/pkg/api package only contains the code we want in the k8s.io/api repo, then we can run a [script](https://github.com/kubernetes/kubernetes/pull/41747/commits/a0015fd1be6c6a239fc40cc3c04ba5ed7b1ab22e#diff-7a2fbb4371972350ee414c6b88aee1c8) to cut the new repo.

Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Record a warning type event

A warning type event should be recorded when failed to calculate
the number of expected pods.

Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Enable service account token lookup by default

Fixes #24167

```release-note
kube-apiserver: --service-account-lookup now defaults to true, requiring the Secret API object containing the token to exist in order for a service account token to be valid. This enables service account tokens to be revoked by deleting the Secret object containing the token.
```

Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Decouple remotecommand

Refactored unversioned/remotecommand to decouple it from undesirable dependencies:

- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module

Automatic merge from submit-queue

CRI: Stop following container log when container exited.

Fixes https://github.com/kubernetes/kubernetes/issues/44340.

This PR changed kubelet to periodically check whether container is running when following container logs, and stop following when container exited.

I've tried this PR in my local cluster:
```
Wed Apr 12 20:23:54 UTC 2017
Wed Apr 12 20:23:58 UTC 2017
Wed Apr 12 20:24:02 UTC 2017
Wed Apr 12 20:24:06 UTC 2017
Wed Apr 12 20:24:10 UTC 2017
Wed Apr 12 20:24:14 UTC 2017
Wed Apr 12 20:24:18 UTC 2017
Wed Apr 12 20:24:22 UTC 2017
Wed Apr 12 20:24:26 UTC 2017
Wed Apr 12 20:24:30 UTC 2017
Wed Apr 12 20:24:34 UTC 2017
Wed Apr 12 20:24:38 UTC 2017
Wed Apr 12 20:24:42 UTC 2017
Wed Apr 12 20:24:46 UTC 2017
failed to wait logs for log file "/var/log/pods/1d54634c7b31346fc3219f5e0b7507cc/nginx_0.log": container "b9a17a2c53550c3703ab350d85911743af8bf164a41813544fd08fb9585f7501" is not running (state="CONTAINER_EXITED")
```

The only difference is that `ReadLogs` will return error when container exits during following. I'm not sure whether we should get rid of it or not.

@yujuhong @feiskyer @JorritSalverda
/cc @kubernetes/sig-node-bugs 

**Release note**:
```release-note
`kubectl logs -f` now stops following when container stops.
```

Automatic merge from submit-queue

fix typos and update an outdated reference in persistent-volume-provi…

…sioning README



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue

fix typos

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Adjust the link to the right troubleshooting doc page

see #3366

https://github.com/kubernetes/kubernetes.github.io/issues/3366



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

Clean up pre-ControllerRef compatibility logic

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43323 

**Special notes for your reviewer**:
No
**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

'workqueue' modify 'work queue'

Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

variable 'controller' collides with imported package name

variable 'controller' collides with imported package name,
 'controller'  modify 'ctrl'