{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Amazon SageMaker Model Monitor\n",
    "\n",
    "본 노트북은 [링크로 연결된 SageMaker Model Monitor 예제](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker_model_monitor/introduction/SageMaker-ModelMonitoring.html)의 한글번역이며 아래 사항을 다루고 있습니다.:\n",
    "* Amazon SageMaker에서 기계 학습 모델을 호스팅하고 추론 요청, 결과 및 메타데이터 캡처하는 방법\n",
    "* 훈련 단계에서의 데이터 세트를 분석하여 데이터에 대한 basline 제약 조건 생성\n",
    "* 호스팅되는 SageMaker 엔드포인트로부터 baseline 제약 조건을 위반하는 추론데이터 라이브 모니터링 \n",
    "     \n",
    "---\n",
    "## 배경\n",
    "\n",
    "Amazon SageMaker는 모든 개발자와 데이터 과학자에게 기계 학습 모델을 빠르게 구축, 교육 및 배포할 수 있는 기능을 제공합니다. Amazon SageMaker는 전체 기계 학습 워크플로를 포괄하는 완전 관리형 서비스입니다. 데이터에 레이블을 지정하고 준비하고, 알고리즘을 선택하고, 모델을 교육한 다음 배포를 위해 조정 및 최적화할 수 있습니다. Amazon SageMaker를 사용하여 모델을 프로덕션에 배포하여 이전보다 예측하고 비용을 절감할 수 있습니다.\n",
    "\n",
    "또한 Amazon SageMaker를 사용하면 배포하는 모델 호출에 대한 입력, 출력 및 메타데이터를 캡처할 수 있습니다. 또한 데이터를 분석하고 품질을 모니터링할 수 있습니다. 이 노트북에서는 Amazon SageMaker가 이러한 기능을 활성화하는 방법을 살펴봅니다.\n",
    "\n",
    "---\n",
    "## 시작하기\n",
    "\n",
    "시작시 아래 조건이 만족하는지 확인하십시오.\n",
    "* 모델을 호스팅할 AWS 리전을 선택합니다.\n",
    "* Amazon SageMaker에서 Amazon Simple Storage Service(Amazon S3)의 데이터에 대한 액세스 권한을 가지는지 확인합니다. \n",
    "* 모델 훈련에 사용되는 데이터, 추가 모델 데이터 및 모델 호출에서 캡처한 데이터를 저장하기 위한 S3 버킷을 생성합니다. 본 예제는 데모를 위해 동일한 버킷을 사용하고 있지만 실제로는 서로 다른 보안 정책으로 이들을 분리할 수 있습니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "isConfigCell": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RoleArn: arn:aws:iam::095389425207:role/SC-095389425207-pp-ppbl7kvl-SageMakerExecutionRole-O99WVTKX81MU\n",
      "Demo Bucket: sagemaker-ap-northeast-2-095389425207\n",
      "Capture path: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/datacapture\n",
      "Report path: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/reports\n",
      "Preproc Code path: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/code/preprocessor.py\n",
      "Postproc Code path: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/code/postprocessor.py\n",
      "CPU times: user 728 ms, sys: 118 ms, total: 845 ms\n",
      "Wall time: 1.42 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# Handful of configuration\n",
    "\n",
    "import os\n",
    "import boto3\n",
    "import re\n",
    "import json\n",
    "from sagemaker import get_execution_role, session\n",
    "\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "role = get_execution_role()\n",
    "print(\"RoleArn: {}\".format(role))\n",
    "\n",
    "# You can use a different bucket, but make sure the role you chose for this notebook\n",
    "# has the s3:PutObject permissions. This is the bucket into which the data is captured\n",
    "bucket = session.Session(boto3.Session()).default_bucket()\n",
    "print(\"Demo Bucket: {}\".format(bucket))\n",
    "prefix = \"sagemaker/DEMO-ModelMonitor\"\n",
    "\n",
    "data_capture_prefix = \"{}/datacapture\".format(prefix)\n",
    "s3_capture_upload_path = \"s3://{}/{}\".format(bucket, data_capture_prefix)\n",
    "reports_prefix = \"{}/reports\".format(prefix)\n",
    "s3_report_path = \"s3://{}/{}\".format(bucket, reports_prefix)\n",
    "code_prefix = \"{}/code\".format(prefix)\n",
    "s3_code_preprocessor_uri = \"s3://{}/{}/{}\".format(bucket, code_prefix, \"preprocessor.py\")\n",
    "s3_code_postprocessor_uri = \"s3://{}/{}/{}\".format(bucket, code_prefix, \"postprocessor.py\")\n",
    "\n",
    "print(\"Capture path: {}\".format(s3_capture_upload_path))\n",
    "print(\"Report path: {}\".format(s3_report_path))\n",
    "print(\"Preproc Code path: {}\".format(s3_code_preprocessor_uri))\n",
    "print(\"Postproc Code path: {}\".format(s3_code_postprocessor_uri))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 노트북이 이후 단계의 작업을 진행하는 데 필요한 권한이 있는지 빠르게 확인할 수 있습니다. 위에서 지정한 S3 버킷에 간단한 테스트 객체를 넣어봅니다. 이 명령이 실패하면 버킷에 대한 `s3:PutObject` 권한을 갖도록 IAM 역할을 업데이트하고 다시 시도하십시오."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success! You are all set to proceed.\n"
     ]
    }
   ],
   "source": [
    "# Upload some test files\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(\"test_upload/test.txt\").upload_file(\n",
    "    \"test_data/upload-test-file.txt\"\n",
    ")\n",
    "print(\"Success! You are all set to proceed.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PART A: SageMaker 엔드포인트로부터 실시간 추론 데이터 캡처하기 \n",
    "SageMaker 엔드포인트에 데이터 캡처 기능을 추가하고 이를 모니터링해 보겠습니다. \n",
    "\n",
    "### 사전 훈련된 모델을 Amazon S3에 업로드 \n",
    "이 코드는 이미 학습이 완료된 XGBoost 모델을 S3에 업로드한 후 엔드포인트로 배포할 것입니다. 이 모델은 SageMaker의 XGB Churn Prediction Notebook을 사용하여 학습되었습니다. Amazon S3에 이미 사전 훈련된 모델이 있다면 해당 모델을 사용할 수도 있습니다. 이 경우 s3_key를 지정하여 해당 모델로 교체합니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_file = open(\"model/xgb-churn-prediction-model.tar.gz\", \"rb\")\n",
    "s3_key = os.path.join(prefix, \"xgb-churn-prediction-model.tar.gz\")\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(s3_key).upload_fileobj(model_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델을 Amazon SageMaker로 배포하기\n",
    "\n",
    "먼저 SageMaker Endpoint를 만들겠습니다.\n",
    "Churn Prediction 모델을 가리키는 url과 XGBoost 이미지를 이용하여 모델 객체를 만듭니다. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import gmtime, strftime\n",
    "from sagemaker.model import Model\n",
    "from sagemaker.image_uris import retrieve\n",
    "\n",
    "model_name = \"DEMO-xgb-churn-pred-model-monitor-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "model_url = \"https://{}.s3-{}.amazonaws.com/{}/xgb-churn-prediction-model.tar.gz\".format(\n",
    "    bucket, region, prefix\n",
    ")\n",
    "\n",
    "image_uri = retrieve(\"xgboost\", boto3.Session().region_name, \"0.90-1\")\n",
    "\n",
    "model = Model(image_uri=image_uri, model_data=model_url, role=role)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "모델 데이터 품질을 모니터링하는 데이터 캡처를 활성화하기 위해 `DataCaptureConfig`라는 새 캡처 옵션을 지정합니다. 이 구성으로 요청 페이로드, 응답 페이로드 또는 둘 다를 캡처할 수 있습니다. 캡처 구성은 모든 `variants`에 적용됩니다. 그리고 이 설정을 이용하여 모델 배포를 진행합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "EndpointName=DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43\n",
      "---------------!"
     ]
    }
   ],
   "source": [
    "from sagemaker.model_monitor import DataCaptureConfig\n",
    "\n",
    "endpoint_name = \"DEMO-xgb-churn-pred-model-monitor-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "print(\"EndpointName={}\".format(endpoint_name))\n",
    "\n",
    "data_capture_config = DataCaptureConfig(\n",
    "    enable_capture=True, sampling_percentage=100, destination_s3_uri=s3_capture_upload_path\n",
    ")\n",
    "\n",
    "predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=\"ml.m4.xlarge\",\n",
    "    endpoint_name=endpoint_name,\n",
    "    data_capture_config=data_capture_config,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 배포된 모델의 호출\n",
    "\n",
    "이제 이 엔드포인트로 추론 데이터를 보내고 실시간으로 추론할 수 있습니다. 이전 단계에서 데이터 캡처를 활성화했기 때문에 요청 및 응답 페이로드는 일부 추가 메타데이터와 함께 DataCaptureConfig에서 지정한 Amazon Simple Storage Service(Amazon S3) 버킷에 저장됩니다.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 단계는 약 3분 동안 샘플 데이터를 이용하여 SageMaker 엔드포인트를 호출합니다. 앞서 지정한 샘플링 비율에 따라 데이터가 캡처되고 데이터 캡처 옵션이 해제될 때까지 캡처가 계속됩니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending test traffic to the endpoint DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43. \n",
      "Please wait...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.serializers import CSVSerializer\n",
    "import time\n",
    "\n",
    "predictor = Predictor(endpoint_name=endpoint_name, serializer=CSVSerializer())\n",
    "\n",
    "# get a subset of test data for a quick test\n",
    "!head -180 test_data/test-dataset-input-cols.csv > test_data/test_sample.csv\n",
    "print(\"Sending test traffic to the endpoint {}. \\nPlease wait...\".format(endpoint_name))\n",
    "\n",
    "with open(\"test_data/test_sample.csv\", \"r\") as f:\n",
    "    for row in f:\n",
    "        payload = row.rstrip(\"\\n\")\n",
    "        response = predictor.predict(data=payload)\n",
    "        time.sleep(1)\n",
    "\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 캡처된 데이터 확인\n",
    "\n",
    "이제 Amazon S3에 저장된 데이터 캡처 파일을 리스트업해 봅니다. 호출이 발생한 시간을 기준으로 서로 다른 파일이 표시될 것입니다. Amazon S3 경로의 형식은 다음과 같습니다.\n",
    "\n",
    "`s3://{destination-bucket-prefix}/{endpoint-name}/{variant-name}/yyyy/mm/dd/hh/filename.jsonl`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found Capture Files:\n",
      "sagemaker/DEMO-ModelMonitor/datacapture/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/AllTraffic/2021/11/08/12/48-44-057-5d1da6d1-0f64-4206-bc18-620308e0b0c6.jsonl\n",
      " sagemaker/DEMO-ModelMonitor/datacapture/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/AllTraffic/2021/11/08/12/49-44-989-5c8d94b0-6571-4108-873d-2950ba7df4e6.jsonl\n"
     ]
    }
   ],
   "source": [
    "s3_client = boto3.Session().client(\"s3\")\n",
    "current_endpoint_capture_prefix = \"{}/{}\".format(data_capture_prefix, endpoint_name)\n",
    "result = s3_client.list_objects(Bucket=bucket, Prefix=current_endpoint_capture_prefix)\n",
    "capture_files = [capture_file.get(\"Key\") for capture_file in result.get(\"Contents\")]\n",
    "print(\"Found Capture Files:\")\n",
    "print(\"\\n \".join(capture_files))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음으로 단일 캡처 파일의 내용을 봅니다. 캡처된 모든 데이터가 JSON 라인 형식으로 표시되어야 합니다. 캡처된 파일의 처음 몇 줄을 빠르게 살펴보십시오."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"captureData\":{\"endpointInput\":{\"observedContentType\":\"text/csv\",\"mode\":\"INPUT\",\"data\":\"130,0,263.7,113,186.5,103,195.3,99,18.3,6,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0\",\"encoding\":\"CSV\"},\"endpointOutput\":{\"observedContentType\":\"text/csv; charset=utf-8\",\"mode\":\"OUTPUT\",\"data\":\"0.3791300654411316\",\"encoding\":\"CSV\"}},\"eventMetadata\":{\"eventId\":\"ad8a4b12-6f2e-4532-bbb5-1a8309bc5ac4\",\"inferenceTime\":\"2021-11-08T12:49:44Z\"},\"eventVersion\":\"0\"}\n",
      "{\"captureData\":{\"endpointInput\":{\"observedContentType\":\"text/csv\",\"mode\":\"INPUT\",\"data\":\"85,0,210.3,66,195.8,76,221.6,82,11.2,7,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0\",\"encoding\":\"CSV\"},\"endpointOutput\":{\"observedContentType\":\"text/csv; charset=utf-8\",\"mode\":\"OUTPUT\",\"data\":\"0.017852725461125374\",\"encoding\":\"CSV\"}},\"eventMetadata\":{\"eventId\":\"100ba175-20a8-473e-acb4-4ca4322787b5\",\"inferenceTime\":\"2021-11-08T12:49:46Z\"},\"eventVersion\":\"0\"}\n",
      "{\"captureData\":{\"endpointInput\":{\"observedContentType\":\"text/csv\",\"mode\":\"INPUT\",\"data\":\"146,0,160.1,63,208.4,112,177.6,98,9.2,3,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,1,1,0\",\"encoding\":\"CSV\"},\"endpointOutput\":{\"observedContentType\":\"text/csv; charset=utf-8\",\"mode\":\"OUTPUT\",\"data\":\"0.13224346935749054\",\"encoding\":\"CSV\"}},\"eventMetadata\":{\"eventId\":\"bd7267a3-1c2b-4a09-a967-44d2bc2f4069\",\"inferenceTime\":\"2021-11-08T12:49:47Z\"},\"eventVersion\":\"0\"}\n",
      "{\"captureData\":{\"endpointInput\":{\"observedContentType\":\"text/csv\",\"mode\":\"INPUT\",\"data\":\"95,0,203.4,96,168.6,61,173.0,105,13.7,3,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,1,0\",\"encoding\":\"CSV\"},\"endpointOutput\":{\"observedContentType\":\"text/csv; charset=utf-8\",\"mode\":\"OUTPUT\",\"data\":\"0.04007643461227417\",\"encoding\":\"CSV\"}},\"eventMetadata\":{\"eventId\":\"8347138e-\n"
     ]
    }
   ],
   "source": [
    "def get_obj_body(obj_key):\n",
    "    return s3_client.get_object(Bucket=bucket, Key=obj_key).get(\"Body\").read().decode(\"utf-8\")\n",
    "\n",
    "\n",
    "capture_file = get_obj_body(capture_files[-1])\n",
    "print(capture_file[:2000])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "조금 더 눈에 띄는 형식으로 살펴봅니다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"captureData\": {\n",
      "    \"endpointInput\": {\n",
      "      \"observedContentType\": \"text/csv\",\n",
      "      \"mode\": \"INPUT\",\n",
      "      \"data\": \"130,0,263.7,113,186.5,103,195.3,99,18.3,6,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0\",\n",
      "      \"encoding\": \"CSV\"\n",
      "    },\n",
      "    \"endpointOutput\": {\n",
      "      \"observedContentType\": \"text/csv; charset=utf-8\",\n",
      "      \"mode\": \"OUTPUT\",\n",
      "      \"data\": \"0.3791300654411316\",\n",
      "      \"encoding\": \"CSV\"\n",
      "    }\n",
      "  },\n",
      "  \"eventMetadata\": {\n",
      "    \"eventId\": \"ad8a4b12-6f2e-4532-bbb5-1a8309bc5ac4\",\n",
      "    \"inferenceTime\": \"2021-11-08T12:49:44Z\"\n",
      "  },\n",
      "  \"eventVersion\": \"0\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "print(json.dumps(json.loads(capture_file.split(\"\\n\")[0]), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "각 추론요청은 jsonl 파일에서 하나의 line으로 표시되어 있습니다. 해당 라인은 입력과 출력을 모두 포함합니다. 예제에서 엔드포인트 호출 ContentType이 `text/csv` 로 지정되었기 때문에 `observedContentType` 속성값이 이를 반영하고 있고 `encoding` 항목에서도 마찬가지로 적용된 형식을 알려주고 있습니다. \n",
    "\n",
    "본 단계에서는 엔드포인트에 대한 입력과 출력 페이로드를 어떻게 캡처하는지 살펴보았습니다. 그리고 캡처한 데이터의 형식이 S3에 어떻게 저장되는지 살펴보았습니다. 다음 단계에서는 이렇게 수집한 데이터를 모니터링하고자 할 때 SageMaker가 어떻게 도움을 줄 수 있는지 살펴보겠습니다. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PART B: Model Monitor - 기준선(Baseline) 설정과 지속적인 모니터링 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "추론 데이터 수집 외에도 Amazon SageMaker는 엔드포인트에서 캡처한 데이터를 모니터링하고 평가할 수 있는 기능을 제공합니다. 이를 위해:\n",
    "\n",
    "1. 실시간 트래픽을 비교할 기준선(baseline)이 필요합니다.\n",
    "2. 기준선이 준비되면 스케줄을 설정하여 기준선을 지속적으로 평가하고 비교합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 기준선/학습 데이터 세트를 사용하여 제약 조건 제안"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "모델을 훈련한 훈련 데이터셋은 일반적으로 잘 정제된 데이터들입니다. 그리고 훈련 데이터셋의 스키마와 추론 데이터셋의 스키마는 정확히 일치해야 합니다(i.e. 기능의 수, 순서 등).\n",
    "\n",
    "Amazon SageMaker는 학습단계에서 훈련 데이터셋을 탐색하여 기술 통계(`statistics`)를 생성고 이를 이용하여 모니터링용 제약 조건(`constraints`)를 제안하는 기능을 제공합니다. 본 예제에서는 다음 코드를 이용하여 학습에 사용한 훈련 데이터셋을 S3에 업로드하겠습니다. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline data uri: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/data\n",
      "Baseline results uri: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/results\n"
     ]
    }
   ],
   "source": [
    "# copy over the training dataset to Amazon S3 (if you already have it in Amazon S3, you could reuse it)\n",
    "baseline_prefix = prefix + \"/baselining\"\n",
    "baseline_data_prefix = baseline_prefix + \"/data\"\n",
    "baseline_results_prefix = baseline_prefix + \"/results\"\n",
    "\n",
    "baseline_data_uri = \"s3://{}/{}\".format(bucket, baseline_data_prefix)\n",
    "baseline_results_uri = \"s3://{}/{}\".format(bucket, baseline_results_prefix)\n",
    "print(\"Baseline data uri: {}\".format(baseline_data_uri))\n",
    "print(\"Baseline results uri: {}\".format(baseline_results_uri))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data_file = open(\"test_data/training-dataset-with-header.csv\", \"rb\")\n",
    "s3_key = os.path.join(baseline_prefix, \"data\", \"training-dataset-with-header.csv\")\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(s3_key).upload_fileobj(training_data_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 훈련 데이터셋에 대한 기준선 작업 생성"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 S3에 훈련 데이터가 준비되었으므로 제약 조건을 제안(`suggest`)하는 작업을 시작합니다. `DefaultModelMonitor.suggest_baseline(..)`은 Amazon SageMaker에서 제공하는 Model Monitor 컨테이너를 사용하여 `ProcessingJob`을 시작하고 제약 조건을 생성합니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Job Name:  baseline-suggestion-job-2021-11-08-13-18-16-155\n",
      "Inputs:  [{'InputName': 'baseline_dataset_input', 'AppManaged': False, 'S3Input': {'S3Uri': 's3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/data/training-dataset-with-header.csv', 'LocalPath': '/opt/ml/processing/input/baseline_dataset_input', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3DataDistributionType': 'FullyReplicated', 'S3CompressionType': 'None'}}]\n",
      "Outputs:  [{'OutputName': 'monitoring_output', 'AppManaged': False, 'S3Output': {'S3Uri': 's3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/results', 'LocalPath': '/opt/ml/processing/output', 'S3UploadMode': 'EndOfJob'}}]\n",
      "........................\u001b[34m2021-11-08 13:22:07,806 - matplotlib.font_manager - INFO - Generating new fontManager, this may take some time...\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:08.320004: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:08.320034: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:09.822444: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:09.822473: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:09.822494: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-10-0-243-141.ap-northeast-2.compute.internal): /proc/driver/nvidia/version does not exist\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:09.822744: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\u001b[0m\n",
      "\u001b[34mTo enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:10,976 - __main__ - INFO - All params:{'ProcessingJobArn': 'arn:aws:sagemaker:ap-northeast-2:095389425207:processing-job/baseline-suggestion-job-2021-11-08-13-18-16-155', 'ProcessingJobName': 'baseline-suggestion-job-2021-11-08-13-18-16-155', 'Environment': {'dataset_format': '{\"csv\": {\"header\": true, \"output_columns_position\": \"START\"}}', 'dataset_source': '/opt/ml/processing/input/baseline_dataset_input', 'output_path': '/opt/ml/processing/output', 'publish_cloudwatch_metrics': 'Disabled'}, 'AppSpecification': {'ImageUri': '709848358524.dkr.ecr.ap-northeast-2.amazonaws.com/sagemaker-model-monitor-analyzer', 'ContainerEntrypoint': None, 'ContainerArguments': None}, 'ProcessingInputs': [{'InputName': 'baseline_dataset_input', 'AppManaged': False, 'S3Input': {'LocalPath': '/opt/ml/processing/input/baseline_dataset_input', 'S3Uri': 's3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/data/training-dataset-with-header.csv', 'S3DataDistributionType': 'FullyReplicated', 'S3DataType': 'S3Prefix', 'S3InputMode': 'File', 'S3CompressionType': 'None', 'S3DownloadMode': 'StartOfJob'}, 'DatasetDefinition': None}], 'ProcessingOutputConfig': {'Outputs': [{'OutputName': 'monitoring_output', 'AppManaged': False, 'S3Output': {'LocalPath': '/opt/ml/processing/output', 'S3Uri': 's3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/baselining/results', 'S3UploadMode': 'EndOfJob'}, 'FeatureStoreOutput': None}], 'KmsKeyId': None}, 'ProcessingResources': {'ClusterConfig': {'InstanceCount': 1, 'InstanceType': 'ml.m5.xlarge', 'VolumeSizeInGB': 20, 'VolumeKmsKeyId': None}}, 'RoleArn': 'arn:aws:iam::095389425207:role/SC-095389425207-pp-ppbl7kvl-SageMakerExecutionRole-O99WVTKX81MU', 'StoppingCondition': {'MaxRuntimeInSeconds': 3600}}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:10,976 - __main__ - INFO - Current Environment:{'dataset_format': '{\"csv\": {\"header\": true, \"output_columns_position\": \"START\"}}', 'dataset_source': '/opt/ml/processing/input/baseline_dataset_input', 'output_path': '/opt/ml/processing/output', 'publish_cloudwatch_metrics': 'Disabled'}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:10,976 - DefaultDataAnalyzer - INFO - Performing analysis with input: {\"dataset_source\": \"/opt/ml/processing/input/baseline_dataset_input\", \"dataset_format\": {\"csv\": {\"header\": true, \"output_columns_position\": \"START\"}}, \"output_path\": \"/opt/ml/processing/output\", \"analysis_type\": null, \"problem_type\": null, \"inference_attribute\": null, \"probability_attribute\": null, \"ground_truth_attribute\": null, \"probability_threshold_attribute\": null, \"positive_label\": null, \"record_preprocessor_script\": null, \"post_analytics_processor_script\": null, \"baseline_constraints\": null, \"baseline_statistics\": null, \"start_time\": null, \"end_time\": null, \"metric_time\": null, \"cloudwatch_metrics_directory\": \"/opt/ml/output/metrics/cloudwatch\", \"publish_cloudwatch_metrics\": \"Disabled\", \"sagemaker_endpoint_name\": null, \"sagemaker_monitoring_schedule_name\": null, \"output_message_file\": \"/opt/ml/output/message\", \"detect_outliers\": null, \"detect_drift\": null, \"image_data\": null, \"report_enabled\": false, \"auto_ml_job_detail\": null}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:10,976 - DefaultDataAnalyzer - INFO - Bootstrapping yarn\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:10,976 - bootstrap - INFO - Copy aws jars\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,033 - bootstrap - INFO - Copy cluster config\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,033 - bootstrap - INFO - Write runtime cluster config\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,034 - bootstrap - INFO - Resource Config is: {'current_host': 'algo-1', 'hosts': ['algo-1']}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,041 - bootstrap - INFO - Finished Yarn configuration files setup.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,041 - bootstrap - INFO - Starting spark process for master node algo-1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,041 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/hdfs namenode -format -force\u001b[0m\n",
      "\u001b[34mWARNING: /usr/hadoop-3.0.0/logs does not exist. Creating.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,469 INFO namenode.NameNode: STARTUP_MSG: \u001b[0m\n",
      "\u001b[34m/************************************************************\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG: Starting NameNode\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   host = algo-1/10.0.243.141\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   args = [-format, -force]\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   version = 3.0.0\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   classpath = /usr/hadoop-3.0.0/etc/hadoop:/usr/hadoop-3.0.0/share/hadoop/common/lib/re2j-1.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-webapp-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jersey-json-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/metrics-core-3.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/json-smart-2.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/httpcore-4.4.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-http-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/xz-1.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jersey-core-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-lang3-3.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-io-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-client-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/curator-client-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/stax2-api-3.1.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-security-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/asm-5.0.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-util-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-servlet-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jersey-servlet-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-server-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/hadoop-auth-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/hadoop-annotations-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/curator-recipes-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jsch-0.1.54.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/curator-framework-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/avro-1.7.7.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jackson-core-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/token-provider-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/httpclient-4.5.2.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerby-config-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/netty-3.10.5.Final.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-core-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/junit-4.11.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-server-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jersey-server-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerby-util-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-util-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-common-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/accessors-smart-1.2.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/zookeeper-3.4.9.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/jetty-xml-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/hadoop-aws-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/lib/aws-java-sdk-bundle-1.11.199.jar:/usr/hadoop-3.0.0/share/hadoop/common/hadoop-nfs-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/hadoop-kms-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/common/hadoop-common-3.0.0-tests.jar:/usr/hadoop-3.0.0/share/hadoop/common/hadoop-common-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-webapp-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/json-smart-2.3.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-http-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/xz-1.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-io-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/curator-client-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-security-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/asm-5.0.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-servlet-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-server-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/hadoop-auth-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/hadoop-annotations-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/gson-2.2.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/curator-recipes-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/curator-framework-2.12.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/avro-1.7.7.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/okio-1.4.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jettison-1.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/paranamer-2.3.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-util-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-net-3.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-compress-1.4.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/okhttp-2.4.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/zookeeper-3.4.9.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/lib/jetty-xml-9.3.19.v20170502.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-client-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-3.0.0-tests.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.0.0-tests.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-client-3.0.0-tests.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-tests.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jcodings-1.0.8.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/metrics-core-3.0.1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jasper-runtime-5.5.23.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jsp-api-2.1-6.1.14.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jersey-client-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-server-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jamon-runtime-2.4.1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-annotations-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/fst-2.50.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/htrace-core-3.1.0-incubating.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-procedure-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/metrics-core-2.2.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/commons-math-2.2.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-hadoop2-compat-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-hadoop-compat-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/findbugs-annotations-1.3.9-1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-protocol-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/disruptor-3.3.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-prefix-tree-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/servlet-api-2.5-6.1.14.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jsp-2.1-6.1.14.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/java-util-1.9.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-client-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/hbase-common-1.2.6.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/commons-el-1.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/joni-2.1.2.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/json-io-2.5.1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/commons-csv-1.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/jasper-compiler-5.5.23.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-timelineservice-hbase-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-timelineservice-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-common-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.0.0.jar:/usr/hadoop-\u001b[0m\n",
      "\u001b[34m3.0.0/share/hadoop/yarn/hadoop-yarn-server-timelineservice-hbase-tests-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-registry-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-router-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-client-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-api-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.0.0.jar:/usr/hadoop-3.0.0/share/hadoop/yarn/hadoop-yarn-server-common-3.0.0.jar\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r c25427ceca461ee979d30edd7a4b0f50718e6533; compiled by 'andrew' on 2017-12-08T19:16Z\u001b[0m\n",
      "\u001b[34mSTARTUP_MSG:   java = 1.8.0_302\u001b[0m\n",
      "\u001b[34m************************************************************/\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,478 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,482 INFO namenode.NameNode: createNameNode [-format, -force]\u001b[0m\n",
      "\u001b[34mFormatting using clusterid: CID-c78cebab-29a6-4a2e-b271-a03225aaf30e\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,952 INFO namenode.FSEditLog: Edit logging is async:true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,962 INFO namenode.FSNamesystem: KeyProvider: null\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,963 INFO namenode.FSNamesystem: fsLock is fair: true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,965 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,970 INFO namenode.FSNamesystem: fsOwner             = root (auth:SIMPLE)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,970 INFO namenode.FSNamesystem: supergroup          = supergroup\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,970 INFO namenode.FSNamesystem: isPermissionEnabled = true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,970 INFO namenode.FSNamesystem: HA Enabled: false\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:11,999 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,009 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,012 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,015 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,016 INFO blockmanagement.BlockManager: The block deletion will start around 2021 Nov 08 13:22:12\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,017 INFO util.GSet: Computing capacity for map BlocksMap\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,017 INFO util.GSet: VM type       = 64-bit\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,018 INFO util.GSet: 2.0% max memory 3.1 GB = 63.3 MB\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,018 INFO util.GSet: capacity      = 2^23 = 8388608 entries\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,086 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManager: defaultReplication         = 3\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManager: maxReplication             = 512\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManager: minReplication             = 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,090 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,091 INFO blockmanagement.BlockManager: encryptDataTransfer        = false\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,091 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,115 INFO util.GSet: Computing capacity for map INodeMap\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,115 INFO util.GSet: VM type       = 64-bit\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,116 INFO util.GSet: 1.0% max memory 3.1 GB = 31.6 MB\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,116 INFO util.GSet: capacity      = 2^22 = 4194304 entries\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,118 INFO namenode.FSDirectory: ACLs enabled? false\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,118 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,118 INFO namenode.FSDirectory: XAttrs enabled? true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,118 INFO namenode.NameNode: Caching file names occurring more than 10 times\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,122 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,125 INFO util.GSet: Computing capacity for map cachedBlocks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,125 INFO util.GSet: VM type       = 64-bit\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,126 INFO util.GSet: 0.25% max memory 3.1 GB = 7.9 MB\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,126 INFO util.GSet: capacity      = 2^20 = 1048576 entries\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,132 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,132 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,132 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,135 INFO namenode.FSNamesystem: Retry cache on namenode is enabled\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,135 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,137 INFO util.GSet: Computing capacity for map NameNodeRetryCache\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,137 INFO util.GSet: VM type       = 64-bit\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,137 INFO util.GSet: 0.029999999329447746% max memory 3.1 GB = 971.7 KB\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,137 INFO util.GSet: capacity      = 2^17 = 131072 entries\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,159 INFO namenode.FSImage: Allocated new BlockPoolId: BP-357361952-10.0.243.141-1636377732151\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,170 INFO common.Storage: Storage directory /opt/amazon/hadoop/hdfs/namenode has been successfully formatted.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,177 INFO namenode.FSImageFormatProtobuf: Saving image file /opt/amazon/hadoop/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 using no compression\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,251 INFO namenode.FSImageFormatProtobuf: Image file /opt/amazon/hadoop/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 of size 389 bytes saved in 0 seconds.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,261 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,264 INFO namenode.NameNode: SHUTDOWN_MSG: \u001b[0m\n",
      "\u001b[34m/************************************************************\u001b[0m\n",
      "\u001b[34mSHUTDOWN_MSG: Shutting down NameNode at algo-1/10.0.243.141\u001b[0m\n",
      "\u001b[34m************************************************************/\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:12,273 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/hdfs --daemon start namenode\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:14,329 - bootstrap - INFO - Failed to run /usr/hadoop-3.0.0/bin/hdfs --daemon start namenode, return code 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:14,329 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/hdfs --daemon start datanode\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:16,401 - bootstrap - INFO - Failed to run /usr/hadoop-3.0.0/bin/hdfs --daemon start datanode, return code 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:16,402 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/yarn --daemon start resourcemanager\u001b[0m\n",
      "\u001b[34mWARNING: YARN_LOG_DIR has been replaced by HADOOP_LOG_DIR. Using value of YARN_LOG_DIR.\u001b[0m\n",
      "\u001b[34mWARNING: /var/log/yarn/ does not exist. Creating.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:18,495 - bootstrap - INFO - Failed to run /usr/hadoop-3.0.0/bin/yarn --daemon start resourcemanager, return code 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:18,495 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/yarn --daemon start nodemanager\u001b[0m\n",
      "\u001b[34mWARNING: YARN_LOG_DIR has been replaced by HADOOP_LOG_DIR. Using value of YARN_LOG_DIR.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:20,604 - bootstrap - INFO - Failed to run /usr/hadoop-3.0.0/bin/yarn --daemon start nodemanager, return code 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:20,604 - bootstrap - INFO - Running command: /usr/hadoop-3.0.0/bin/yarn --daemon start proxyserver\u001b[0m\n",
      "\u001b[34mWARNING: YARN_LOG_DIR has been replaced by HADOOP_LOG_DIR. Using value of YARN_LOG_DIR.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:22,671 - bootstrap - INFO - Failed to run /usr/hadoop-3.0.0/bin/yarn --daemon start proxyserver, return code 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:22,672 - DefaultDataAnalyzer - INFO - Total number of hosts in the cluster: 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:32,683 - DefaultDataAnalyzer - INFO - Running command: bin/spark-submit --master yarn --deploy-mode client --conf spark.hadoop.fs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider --conf spark.serializer=org.apache.spark.serializer.KryoSerializer /opt/amazon/sagemaker-data-analyzer-1.0-jar-with-dependencies.jar --analytics_input /tmp/spark_job_config.json\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:33 WARN  NativeCodeLoader:60 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:33 INFO  Main:28 - Start analyzing with args: --analytics_input /tmp/spark_job_config.json\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:33 INFO  Main:31 - Analytics input path: DataAnalyzerParams(/tmp/spark_job_config.json,yarn)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:33 INFO  FileUtil:66 - Read file from path /tmp/spark_job_config.json.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkContext:54 - Running Spark version 2.3.1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkContext:54 - Submitted application: SageMakerDataAnalyzer\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SecurityManager:54 - Changing view acls to: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SecurityManager:54 - Changing modify acls to: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SecurityManager:54 - Changing view acls groups to: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SecurityManager:54 - Changing modify acls groups to: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 45973.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkEnv:54 - Registering MapOutputTracker\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkEnv:54 - Registering BlockManagerMaster\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-37c5224e-155b-46c6-a947-0710912696fb\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  MemoryStore:54 - MemoryStore started with capacity 1458.6 MB\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkEnv:54 - Registering OutputCommitCoordinator\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:34 INFO  SparkContext:54 - Added JAR file:/opt/amazon/sagemaker-data-analyzer-1.0-jar-with-dependencies.jar at spark://10.0.243.141:45973/jars/sagemaker-data-analyzer-1.0-jar-with-dependencies.jar with timestamp 1636377754758\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  RMProxy:133 - Connecting to ResourceManager at /10.0.243.141:8032\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Requesting a new application from cluster with 1 NodeManagers\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Configuration:2636 - resource-types.xml not found\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  ResourceUtils:427 - Unable to find 'resource-types.xml'.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Verifying our application has not requested more than the maximum memory capability of the cluster (15575 MB per container)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Will allocate AM container, with 896 MB memory including 384 MB overhead\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Setting up container launch context for our AM\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Setting up the launch environment for our AM container\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:35 INFO  Client:54 - Preparing resources for our AM container\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:36 WARN  Client:66 - Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:38 INFO  Client:54 - Uploading resource file:/tmp/spark-b51fc04a-9ef0-4ddc-9c9a-1fd6503f2b3f/__spark_libs__153866718783192760.zip -> hdfs://10.0.243.141/user/root/.sparkStaging/application_1636377737778_0001/__spark_libs__153866718783192760.zip\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  Client:54 - Uploading resource file:/tmp/spark-b51fc04a-9ef0-4ddc-9c9a-1fd6503f2b3f/__spark_conf__3321510594509558784.zip -> hdfs://10.0.243.141/user/root/.sparkStaging/application_1636377737778_0001/__spark_conf__.zip\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SecurityManager:54 - Changing view acls to: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SecurityManager:54 - Changing modify acls to: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SecurityManager:54 - Changing view acls groups to: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SecurityManager:54 - Changing modify acls groups to: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  Client:54 - Submitting application application_1636377737778_0001 to ResourceManager\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  YarnClientImpl:310 - Submitted application application_1636377737778_0001\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:39 INFO  SchedulerExtensionServices:54 - Starting Yarn extension services with app application_1636377737778_0001 and attemptId None\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:40 INFO  Client:54 - Application report for application_1636377737778_0001 (state: ACCEPTED)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:40 INFO  Client:54 - \u001b[0m\n",
      "\u001b[34m#011 client token: N/A\u001b[0m\n",
      "\u001b[34m#011 diagnostics: AM container is launched, waiting for AM container to Register with RM\u001b[0m\n",
      "\u001b[34m#011 ApplicationMaster host: N/A\u001b[0m\n",
      "\u001b[34m#011 ApplicationMaster RPC port: -1\u001b[0m\n",
      "\u001b[34m#011 queue: default\u001b[0m\n",
      "\u001b[34m#011 start time: 1636377759421\u001b[0m\n",
      "\u001b[34m#011 final status: UNDEFINED\u001b[0m\n",
      "\u001b[34m#011 tracking URL: http://algo-1:8088/proxy/application_1636377737778_0001/\u001b[0m\n",
      "\u001b[34m#011 user: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:41 INFO  Client:54 - Application report for application_1636377737778_0001 (state: ACCEPTED)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:42 INFO  Client:54 - Application report for application_1636377737778_0001 (state: ACCEPTED)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:43 INFO  Client:54 - Application report for application_1636377737778_0001 (state: ACCEPTED)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:43 INFO  YarnClientSchedulerBackend:54 - Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> algo-1, PROXY_URI_BASES -> http://algo-1:8088/proxy/application_1636377737778_0001), /proxy/application_1636377737778_0001\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  YarnSchedulerBackend$YarnSchedulerEndpoint:54 - ApplicationMaster registered as NettyRpcEndpointRef(spark-client://YarnAM)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  Client:54 - Application report for application_1636377737778_0001 (state: RUNNING)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  Client:54 - \u001b[0m\n",
      "\u001b[34m#011 client token: N/A\u001b[0m\n",
      "\u001b[34m#011 diagnostics: N/A\u001b[0m\n",
      "\u001b[34m#011 ApplicationMaster host: 10.0.243.141\u001b[0m\n",
      "\u001b[34m#011 ApplicationMaster RPC port: 0\u001b[0m\n",
      "\u001b[34m#011 queue: default\u001b[0m\n",
      "\u001b[34m#011 start time: 1636377759421\u001b[0m\n",
      "\u001b[34m#011 final status: UNDEFINED\u001b[0m\n",
      "\u001b[34m#011 tracking URL: http://algo-1:8088/proxy/application_1636377737778_0001/\u001b[0m\n",
      "\u001b[34m#011 user: root\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  YarnClientSchedulerBackend:54 - Application application_1636377737778_0001 has started running.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 40637.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  NettyBlockTransferService:54 - Server created on 10.0.243.141:40637\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, 10.0.243.141, 40637, None)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  BlockManagerMasterEndpoint:54 - Registering block manager 10.0.243.141:40637 with 1458.6 MB RAM, BlockManagerId(driver, 10.0.243.141, 40637, None)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, 10.0.243.141, 40637, None)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, 10.0.243.141, 40637, None)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:44 INFO  log:192 - Logging initialized @11731ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:47 INFO  YarnSchedulerBackend$YarnDriverEndpoint:54 - Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.0.243.141:45068) with ID 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:22:47 INFO  BlockManagerMasterEndpoint:54 - Registering block manager algo-1:36955 with 5.8 GB RAM, BlockManagerId(1, algo-1, 36955, None)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:04 INFO  YarnClientSchedulerBackend:54 - SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000(ms)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:05 WARN  SparkContext:66 - Spark is not running in local mode, therefore the checkpoint directory must not be on the local filesystem. Directory '/tmp' appears to be on the local filesystem.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:05 INFO  DatasetReader:97 - Files to process:List(file:///opt/ml/processing/input/baseline_dataset_input/training-dataset-with-header.csv)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:05 INFO  SharedState:54 - Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/usr/spark-2.3.1/spark-warehouse').\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:05 INFO  SharedState:54 - Warehouse path is 'file:/usr/spark-2.3.1/spark-warehouse'.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:05 INFO  StateStoreCoordinatorRef:54 - Registered StateStoreCoordinator endpoint\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  FileSourceStrategy:54 - Pruning directories with: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  FileSourceStrategy:54 - Post-Scan Filters: (length(trim(value#0, None)) > 0)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  FileSourceStrategy:54 - Output Data Schema: struct<value: string>\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  FileSourceScanExec:54 - Pushed Filters: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  CodeGenerator:54 - Code generated in 179.635665 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  CodeGenerator:54 - Code generated in 21.853362 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  MemoryStore:54 - Block broadcast_0 stored as values in memory (estimated size 429.7 KB, free 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  MemoryStore:54 - Block broadcast_0_piece0 stored as bytes in memory (estimated size 38.3 KB, free 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  BlockManagerInfo:54 - Added broadcast_0_piece0 in memory on 10.0.243.141:40637 (size: 38.3 KB, free: 1458.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  SparkContext:54 - Created broadcast 0 from csv at DatasetReader.scala:51\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:06 INFO  FileSourceScanExec:54 - Planning scan with bin packing, max size: 4194304 bytes, open cost is considered as scanning 4194304 bytes.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  SparkContext:54 - Starting job: csv at DatasetReader.scala:51\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Got job 0 (csv at DatasetReader.scala:51) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Final stage: ResultStage 0 (csv at DatasetReader.scala:51)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Submitting ResultStage 0 (MapPartitionsRDD[3] at csv at DatasetReader.scala:51), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  MemoryStore:54 - Block broadcast_1 stored as values in memory (estimated size 9.4 KB, free 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  MemoryStore:54 - Block broadcast_1_piece0 stored as bytes in memory (estimated size 4.5 KB, free 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  BlockManagerInfo:54 - Added broadcast_1_piece0 in memory on 10.0.243.141:40637 (size: 4.5 KB, free: 1458.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  SparkContext:54 - Created broadcast 1 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at csv at DatasetReader.scala:51) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  YarnScheduler:54 - Adding task set 0.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:07 INFO  TaskSetManager:54 - Starting task 0.0 in stage 0.0 (TID 0, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:08 INFO  BlockManagerInfo:54 - Added broadcast_1_piece0 in memory on algo-1:36955 (size: 4.5 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:08 INFO  BlockManagerInfo:54 - Added broadcast_0_piece0 in memory on algo-1:36955 (size: 38.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  TaskSetManager:54 - Finished task 0.0 in stage 0.0 (TID 0) in 1857 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  YarnScheduler:54 - Removed TaskSet 0.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - ResultStage 0 (csv at DatasetReader.scala:51) finished in 1.950 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Job 0 finished: csv at DatasetReader.scala:51, took 1.996483 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Pruning directories with: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Post-Scan Filters: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Output Data Schema: struct<value: string>\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceScanExec:54 - Pushed Filters: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  CodeGenerator:54 - Code generated in 8.428296 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_2 stored as values in memory (estimated size 429.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 9\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 18\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 17\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 14\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 6\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 29\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 12\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 4\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 20\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 28\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_2_piece0 stored as bytes in memory (estimated size 38.3 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Added broadcast_2_piece0 in memory on 10.0.243.141:40637 (size: 38.3 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Removed broadcast_0_piece0 on algo-1:36955 in memory (size: 38.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  SparkContext:54 - Created broadcast 2 from csv at DatasetReader.scala:51\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceScanExec:54 - Planning scan with bin packing, max size: 4194304 bytes, open cost is considered as scanning 4194304 bytes.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Removed broadcast_0_piece0 on 10.0.243.141:40637 in memory (size: 38.3 KB, free: 1458.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 3\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 15\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 25\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 21\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Removed broadcast_1_piece0 on 10.0.243.141:40637 in memory (size: 4.5 KB, free: 1458.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Removed broadcast_1_piece0 on algo-1:36955 in memory (size: 4.5 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 19\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 7\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 13\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 10\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 27\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 16\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 2\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 22\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 24\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 30\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 5\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 26\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 0\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 8\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 23\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  ContextCleaner:54 - Cleaned accumulator 11\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Pruning directories with: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Post-Scan Filters: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceStrategy:54 - Output Data Schema: struct<Churn: string, Account Length: string, VMail Message: string, Day Mins: string, Day Calls: string ... 68 more fields>\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceScanExec:54 - Pushed Filters: \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_3 stored as values in memory (estimated size 429.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_3_piece0 stored as bytes in memory (estimated size 38.3 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Added broadcast_3_piece0 in memory on 10.0.243.141:40637 (size: 38.3 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  SparkContext:54 - Created broadcast 3 from cache at DataAnalyzer.scala:78\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  FileSourceScanExec:54 - Planning scan with bin packing, max size: 4194304 bytes, open cost is considered as scanning 4194304 bytes.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 WARN  Utils:66 - Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  CodeGenerator:54 - Code generated in 94.491828 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  CodeGenerator:54 - Code generated in 63.846992 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  SparkContext:54 - Starting job: head at DataAnalyzer.scala:81\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Got job 1 (head at DataAnalyzer.scala:81) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Final stage: ResultStage 1 (head at DataAnalyzer.scala:81)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Submitting ResultStage 1 (MapPartitionsRDD[17] at head at DataAnalyzer.scala:81), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_4 stored as values in memory (estimated size 85.3 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  MemoryStore:54 - Block broadcast_4_piece0 stored as bytes in memory (estimated size 27.6 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Added broadcast_4_piece0 in memory on 10.0.243.141:40637 (size: 27.6 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  SparkContext:54 - Created broadcast 4 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[17] at head at DataAnalyzer.scala:81) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  YarnScheduler:54 - Adding task set 1.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  TaskSetManager:54 - Starting task 0.0 in stage 1.0 (TID 1, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:09 INFO  BlockManagerInfo:54 - Added broadcast_4_piece0 in memory on algo-1:36955 (size: 27.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  BlockManagerInfo:54 - Added broadcast_3_piece0 in memory on algo-1:36955 (size: 38.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  BlockManagerInfo:54 - Added rdd_11_0 in memory on algo-1:36955 (size: 188.0 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  TaskSetManager:54 - Finished task 0.0 in stage 1.0 (TID 1) in 842 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  YarnScheduler:54 - Removed TaskSet 1.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  DAGScheduler:54 - ResultStage 1 (head at DataAnalyzer.scala:81) finished in 0.893 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  DAGScheduler:54 - Job 1 finished: head at DataAnalyzer.scala:81, took 0.900295 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  DataAnalyzer:89 - The number of columns in the dataframe is 70\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  DataAnalyzer:116 - Number of shards is: 14\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:10 INFO  CodeGenerator:54 - Code generated in 13.327893 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Registering RDD 22 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Got job 2 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Final stage: ResultStage 3 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 2)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 2)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 2 (MapPartitionsRDD[22] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  MemoryStore:54 - Block broadcast_5 stored as values in memory (estimated size 108.7 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  MemoryStore:54 - Block broadcast_5_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  BlockManagerInfo:54 - Added broadcast_5_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  SparkContext:54 - Created broadcast 5 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 2 (MapPartitionsRDD[22] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  YarnScheduler:54 - Adding task set 2.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  TaskSetManager:54 - Starting task 0.0 in stage 2.0 (TID 2, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  BlockManagerInfo:54 - Added broadcast_5_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  TaskSetManager:54 - Finished task 0.0 in stage 2.0 (TID 2) in 687 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  YarnScheduler:54 - Removed TaskSet 2.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - ShuffleMapStage 2 (collect at AnalysisRunner.scala:313) finished in 0.710 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - waiting: Set(ResultStage 3)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Submitting ResultStage 3 (MapPartitionsRDD[25] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  MemoryStore:54 - Block broadcast_6 stored as values in memory (estimated size 121.2 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  MemoryStore:54 - Block broadcast_6_piece0 stored as bytes in memory (estimated size 40.9 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  BlockManagerInfo:54 - Added broadcast_6_piece0 in memory on 10.0.243.141:40637 (size: 40.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  SparkContext:54 - Created broadcast 6 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 3 (MapPartitionsRDD[25] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  YarnScheduler:54 - Adding task set 3.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  TaskSetManager:54 - Starting task 0.0 in stage 3.0 (TID 3, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  BlockManagerInfo:54 - Added broadcast_6_piece0 in memory on algo-1:36955 (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:11 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 0 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  TaskSetManager:54 - Finished task 0.0 in stage 3.0 (TID 3) in 288 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - ResultStage 3 (collect at AnalysisRunner.scala:313) finished in 0.303 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  YarnScheduler:54 - Removed TaskSet 3.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Job 2 finished: collect at AnalysisRunner.scala:313, took 1.031922 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  CodeGenerator:54 - Code generated in 15.694365 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Got job 3 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Final stage: ResultStage 4 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Submitting ResultStage 4 (MapPartitionsRDD[34] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  MemoryStore:54 - Block broadcast_7 stored as values in memory (estimated size 69.0 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  MemoryStore:54 - Block broadcast_7_piece0 stored as bytes in memory (estimated size 26.6 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  BlockManagerInfo:54 - Added broadcast_7_piece0 in memory on 10.0.243.141:40637 (size: 26.6 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  SparkContext:54 - Created broadcast 7 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 4 (MapPartitionsRDD[34] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  YarnScheduler:54 - Adding task set 4.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  TaskSetManager:54 - Starting task 0.0 in stage 4.0 (TID 4, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  BlockManagerInfo:54 - Added broadcast_7_piece0 in memory on algo-1:36955 (size: 26.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  TaskSetManager:54 - Finished task 0.0 in stage 4.0 (TID 4) in 223 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  YarnScheduler:54 - Removed TaskSet 4.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - ResultStage 4 (treeReduce at KLLRunner.scala:107) finished in 0.239 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  DAGScheduler:54 - Job 3 finished: treeReduce at KLLRunner.scala:107, took 0.242058 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  CodeGenerator:54 - Code generated in 49.21804 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:12 INFO  CodeGenerator:54 - Code generated in 87.432331 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  CodeGenerator:54 - Code generated in 53.584344 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Registering RDD 39 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Got job 4 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Final stage: ResultStage 6 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 5)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 5)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 5 (MapPartitionsRDD[39] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_8 stored as values in memory (estimated size 108.8 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_8_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_8_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 121\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 116\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 124\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 142\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 158\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 108\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 91\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 78\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 74\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 35\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 61\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 67\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 73\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 120\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 130\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 144\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 148\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 145\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 92\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 127\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 87\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 97\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 159\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 85\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 102\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  SparkContext:54 - Created broadcast 8 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 5 (MapPartitionsRDD[39] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  YarnScheduler:54 - Adding task set 5.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  TaskSetManager:54 - Starting task 0.0 in stage 5.0 (TID 5, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_4_piece0 on 10.0.243.141:40637 in memory (size: 27.6 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_4_piece0 on algo-1:36955 in memory (size: 27.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 45\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 70\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 143\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 84\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_8_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 86\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_5_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_5_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 93\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 64\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 95\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 109\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 156\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 54\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_2_piece0 on 10.0.243.141:40637 in memory (size: 38.3 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 117\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 68\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 128\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 154\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 88\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 125\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 31\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 155\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 150\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 52\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 81\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 153\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 138\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 147\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 114\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 46\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 136\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 139\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 101\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 100\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 43\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 33\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 151\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 57\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 65\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 160\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 75\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 82\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 149\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 69\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 76\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned shuffle 0\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 72\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 62\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 123\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 131\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 119\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 49\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 71\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 161\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_7_piece0 on 10.0.243.141:40637 in memory (size: 26.6 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_7_piece0 on algo-1:36955 in memory (size: 26.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 98\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 58\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 83\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 115\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 118\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 90\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 152\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 113\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 89\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 32\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 99\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 55\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 111\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 129\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 106\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 56\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 110\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 132\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 34\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 112\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 42\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 50\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 60\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 103\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 47\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_6_piece0 on 10.0.243.141:40637 in memory (size: 40.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Removed broadcast_6_piece0 on algo-1:36955 in memory (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 48\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 53\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 157\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 137\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 63\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 146\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 51\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 80\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 94\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 105\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 126\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 122\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 141\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 104\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 66\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 79\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 59\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 77\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 44\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 140\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  ContextCleaner:54 - Cleaned accumulator 96\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  TaskSetManager:54 - Finished task 0.0 in stage 5.0 (TID 5) in 230 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  YarnScheduler:54 - Removed TaskSet 5.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - ShuffleMapStage 5 (collect at AnalysisRunner.scala:313) finished in 0.276 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - waiting: Set(ResultStage 6)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting ResultStage 6 (MapPartitionsRDD[42] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_9 stored as values in memory (estimated size 54.8 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_9_piece0 stored as bytes in memory (estimated size 16.8 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_9_piece0 in memory on 10.0.243.141:40637 (size: 16.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  SparkContext:54 - Created broadcast 9 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 6 (MapPartitionsRDD[42] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  YarnScheduler:54 - Adding task set 6.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  TaskSetManager:54 - Starting task 0.0 in stage 6.0 (TID 6, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_9_piece0 in memory on algo-1:36955 (size: 16.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 1 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  TaskSetManager:54 - Finished task 0.0 in stage 6.0 (TID 6) in 150 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  YarnScheduler:54 - Removed TaskSet 6.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - ResultStage 6 (collect at AnalysisRunner.scala:313) finished in 0.170 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Job 4 finished: collect at AnalysisRunner.scala:313, took 0.453036 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Registering RDD 49 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Got job 5 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Final stage: ResultStage 8 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 7)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 7)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 7 (MapPartitionsRDD[49] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_10 stored as values in memory (estimated size 63.5 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  MemoryStore:54 - Block broadcast_10_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_10_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  SparkContext:54 - Created broadcast 10 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 7 (MapPartitionsRDD[49] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  YarnScheduler:54 - Adding task set 7.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  TaskSetManager:54 - Starting task 0.0 in stage 7.0 (TID 7, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:13 INFO  BlockManagerInfo:54 - Added broadcast_10_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  TaskSetManager:54 - Finished task 0.0 in stage 7.0 (TID 7) in 1079 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  YarnScheduler:54 - Removed TaskSet 7.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - ShuffleMapStage 7 (countByKey at ColumnProfiler.scala:566) finished in 1.114 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - waiting: Set(ResultStage 8)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - Submitting ResultStage 8 (ShuffledRDD[50] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  MemoryStore:54 - Block broadcast_11 stored as values in memory (estimated size 3.2 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  MemoryStore:54 - Block broadcast_11_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  BlockManagerInfo:54 - Added broadcast_11_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  SparkContext:54 - Created broadcast 11 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 8 (ShuffledRDD[50] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  YarnScheduler:54 - Adding task set 8.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  TaskSetManager:54 - Starting task 0.0 in stage 8.0 (TID 8, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  BlockManagerInfo:54 - Added broadcast_11_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 2 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  TaskSetManager:54 - Finished task 0.0 in stage 8.0 (TID 8) in 75 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  YarnScheduler:54 - Removed TaskSet 8.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - ResultStage 8 (countByKey at ColumnProfiler.scala:566) finished in 0.088 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:14 INFO  DAGScheduler:54 - Job 5 finished: countByKey at ColumnProfiler.scala:566, took 1.216874 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Registering RDD 55 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Got job 6 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Final stage: ResultStage 10 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 9)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 9)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 9 (MapPartitionsRDD[55] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_12 stored as values in memory (estimated size 108.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_12_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_12_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  SparkContext:54 - Created broadcast 12 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 9 (MapPartitionsRDD[55] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Adding task set 9.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Starting task 0.0 in stage 9.0 (TID 9, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_12_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Finished task 0.0 in stage 9.0 (TID 9) in 214 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Removed TaskSet 9.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - ShuffleMapStage 9 (collect at AnalysisRunner.scala:313) finished in 0.229 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - waiting: Set(ResultStage 10)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting ResultStage 10 (MapPartitionsRDD[58] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_13 stored as values in memory (estimated size 121.2 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_13_piece0 stored as bytes in memory (estimated size 40.9 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_13_piece0 in memory on 10.0.243.141:40637 (size: 40.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  SparkContext:54 - Created broadcast 13 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 10 (MapPartitionsRDD[58] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Adding task set 10.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Starting task 0.0 in stage 10.0 (TID 10, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_13_piece0 in memory on algo-1:36955 (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 3 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Finished task 0.0 in stage 10.0 (TID 10) in 129 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Removed TaskSet 10.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - ResultStage 10 (collect at AnalysisRunner.scala:313) finished in 0.154 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Job 6 finished: collect at AnalysisRunner.scala:313, took 0.388984 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  CodeGenerator:54 - Code generated in 12.137884 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Got job 7 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Final stage: ResultStage 11 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting ResultStage 11 (MapPartitionsRDD[67] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_14 stored as values in memory (estimated size 68.9 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  MemoryStore:54 - Block broadcast_14_piece0 stored as bytes in memory (estimated size 26.6 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_14_piece0 in memory on 10.0.243.141:40637 (size: 26.6 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  SparkContext:54 - Created broadcast 14 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 11 (MapPartitionsRDD[67] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Adding task set 11.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Starting task 0.0 in stage 11.0 (TID 11, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  BlockManagerInfo:54 - Added broadcast_14_piece0 in memory on algo-1:36955 (size: 26.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  TaskSetManager:54 - Finished task 0.0 in stage 11.0 (TID 11) in 175 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - ResultStage 11 (treeReduce at KLLRunner.scala:107) finished in 0.184 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  YarnScheduler:54 - Removed TaskSet 11.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:15 INFO  DAGScheduler:54 - Job 7 finished: treeReduce at KLLRunner.scala:107, took 0.187745 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  CodeGenerator:54 - Code generated in 43.629893 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  CodeGenerator:54 - Code generated in 39.716293 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Registering RDD 72 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Got job 8 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Final stage: ResultStage 13 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 12)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 12)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 12 (MapPartitionsRDD[72] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_15 stored as values in memory (estimated size 107.8 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_15_piece0 stored as bytes in memory (estimated size 37.4 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_15_piece0 in memory on 10.0.243.141:40637 (size: 37.4 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 15 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 12 (MapPartitionsRDD[72] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 12.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 12.0 (TID 12, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_15_piece0 in memory on algo-1:36955 (size: 37.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Finished task 0.0 in stage 12.0 (TID 12) in 104 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Removed TaskSet 12.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - ShuffleMapStage 12 (collect at AnalysisRunner.scala:313) finished in 0.116 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - waiting: Set(ResultStage 13)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ResultStage 13 (MapPartitionsRDD[75] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_16 stored as values in memory (estimated size 54.3 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_16_piece0 stored as bytes in memory (estimated size 16.7 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_16_piece0 in memory on 10.0.243.141:40637 (size: 16.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 16 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 13 (MapPartitionsRDD[75] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 13.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 13.0 (TID 13, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_16_piece0 in memory on algo-1:36955 (size: 16.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 4 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Finished task 0.0 in stage 13.0 (TID 13) in 72 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Removed TaskSet 13.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - ResultStage 13 (collect at AnalysisRunner.scala:313) finished in 0.090 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Job 8 finished: collect at AnalysisRunner.scala:313, took 0.213604 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Registering RDD 82 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Got job 9 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Final stage: ResultStage 15 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 14)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 14)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 14 (MapPartitionsRDD[82] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_17 stored as values in memory (estimated size 63.5 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_17_piece0 stored as bytes in memory (estimated size 24.6 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_17_piece0 in memory on 10.0.243.141:40637 (size: 24.6 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 17 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 14 (MapPartitionsRDD[82] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 14.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 14.0 (TID 14, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_17_piece0 in memory on algo-1:36955 (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Finished task 0.0 in stage 14.0 (TID 14) in 57 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Removed TaskSet 14.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - ShuffleMapStage 14 (countByKey at ColumnProfiler.scala:566) finished in 0.072 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - waiting: Set(ResultStage 15)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ResultStage 15 (ShuffledRDD[83] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_18 stored as values in memory (estimated size 3.2 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_18_piece0 stored as bytes in memory (estimated size 1922.0 B, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_18_piece0 in memory on 10.0.243.141:40637 (size: 1922.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 18 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 15 (ShuffledRDD[83] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 15.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 15.0 (TID 15, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_18_piece0 in memory on algo-1:36955 (size: 1922.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 5 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Finished task 0.0 in stage 15.0 (TID 15) in 45 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Removed TaskSet 15.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - ResultStage 15 (countByKey at ColumnProfiler.scala:566) finished in 0.065 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Job 9 finished: countByKey at ColumnProfiler.scala:566, took 0.151071 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Registering RDD 88 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Got job 10 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Final stage: ResultStage 17 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 16)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 16)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 16 (MapPartitionsRDD[88] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_19 stored as values in memory (estimated size 108.7 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_19_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_19_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 19 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 16 (MapPartitionsRDD[88] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 16.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 16.0 (TID 16, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_19_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Finished task 0.0 in stage 16.0 (TID 16) in 113 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Removed TaskSet 16.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - ShuffleMapStage 16 (collect at AnalysisRunner.scala:313) finished in 0.124 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - waiting: Set(ResultStage 17)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting ResultStage 17 (MapPartitionsRDD[91] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_20 stored as values in memory (estimated size 121.2 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MemoryStore:54 - Block broadcast_20_piece0 stored as bytes in memory (estimated size 41.0 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_20_piece0 in memory on 10.0.243.141:40637 (size: 41.0 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  SparkContext:54 - Created broadcast 20 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 17 (MapPartitionsRDD[91] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  YarnScheduler:54 - Adding task set 17.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  TaskSetManager:54 - Starting task 0.0 in stage 17.0 (TID 17, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  BlockManagerInfo:54 - Added broadcast_20_piece0 in memory on algo-1:36955 (size: 41.0 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:16 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 6 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Finished task 0.0 in stage 17.0 (TID 17) in 104 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Removed TaskSet 17.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - ResultStage 17 (collect at AnalysisRunner.scala:313) finished in 0.121 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Job 10 finished: collect at AnalysisRunner.scala:313, took 0.250154 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  CodeGenerator:54 - Code generated in 9.142123 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Got job 11 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Final stage: ResultStage 18 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting ResultStage 18 (MapPartitionsRDD[100] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_21 stored as values in memory (estimated size 69.1 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_21_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_21_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Created broadcast 21 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 18 (MapPartitionsRDD[100] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Adding task set 18.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Starting task 0.0 in stage 18.0 (TID 18, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_21_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Finished task 0.0 in stage 18.0 (TID 18) in 119 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - ResultStage 18 (treeReduce at KLLRunner.scala:107) finished in 0.131 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Job 11 finished: treeReduce at KLLRunner.scala:107, took 0.136485 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Removed TaskSet 18.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 241\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 578\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 271\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 274\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 354\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 166\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 372\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 379\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 500\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 510\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 233\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 370\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_20_piece0 on 10.0.243.141:40637 in memory (size: 41.0 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_20_piece0 on algo-1:36955 in memory (size: 41.0 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 440\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 541\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_13_piece0 on 10.0.243.141:40637 in memory (size: 40.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_13_piece0 on algo-1:36955 in memory (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 575\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 420\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 479\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 218\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 378\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 530\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 563\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 204\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 515\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 544\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_18_piece0 on 10.0.243.141:40637 in memory (size: 1922.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_18_piece0 on algo-1:36955 in memory (size: 1922.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  CodeGenerator:54 - Code generated in 67.674577 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 539\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 284\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 287\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 392\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 192\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 447\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 215\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 315\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 300\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 252\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 409\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 189\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 454\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 441\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 231\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 335\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 485\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 216\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 238\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_15_piece0 on algo-1:36955 in memory (size: 37.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_15_piece0 on 10.0.243.141:40637 in memory (size: 37.4 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 261\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 519\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 264\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 538\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 400\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 170\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 543\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 564\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 542\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 453\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 176\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 402\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 548\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 235\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 375\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 442\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 242\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 174\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 351\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 475\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 535\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 555\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 532\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 546\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 322\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 303\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 520\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 202\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 506\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 433\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 341\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 579\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 556\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 496\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 413\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 190\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 280\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 565\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 223\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 407\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 576\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 324\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 254\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 508\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 421\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 373\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 377\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_17_piece0 on algo-1:36955 in memory (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_17_piece0 on 10.0.243.141:40637 in memory (size: 24.6 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 281\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 338\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 163\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 386\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 430\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 178\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 266\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 226\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 229\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 290\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 494\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 328\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 256\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 536\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 427\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 222\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 211\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 329\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 448\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 275\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 558\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 3\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 239\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 288\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 455\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 457\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 237\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 405\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 486\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 299\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 297\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 345\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 212\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 437\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 289\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 293\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 385\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 336\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 317\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 301\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 314\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 464\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 460\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 502\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 291\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 331\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 384\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 374\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_19_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_19_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 230\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 554\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 167\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 244\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 325\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 569\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 570\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 380\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 318\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 177\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 398\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 411\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 419\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 391\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 382\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 444\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 493\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 478\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 135\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 472\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 497\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 228\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 561\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 5\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 243\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 360\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 474\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 312\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 358\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 572\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 320\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 503\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 517\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 304\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 4\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 206\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 265\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 452\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 269\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 412\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 295\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 432\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 365\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 263\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 562\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 285\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 306\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 349\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 207\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 333\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 416\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 550\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 343\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 451\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 438\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 337\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 495\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 368\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 571\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 470\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 462\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 461\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 253\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 367\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_11_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_11_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 446\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 560\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 381\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 162\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 395\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 185\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 249\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 348\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 445\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 414\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 577\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 501\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 369\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 547\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 180\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 184\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 512\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 403\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 171\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 522\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 294\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 257\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 435\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 471\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 173\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 557\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 355\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 232\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 234\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 262\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 323\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 417\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 247\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 394\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 484\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 270\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 213\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 182\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 425\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 439\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 396\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 193\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 399\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 195\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  CodeGenerator:54 - Code generated in 56.939526 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 505\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 408\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_8_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_8_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 191\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 305\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 188\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 499\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 418\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 208\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 567\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 397\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 311\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 436\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 326\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 165\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 186\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 434\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 183\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 406\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 236\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 483\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 286\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 393\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 404\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 531\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 350\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 1\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 364\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 282\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 278\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 2\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 227\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 521\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 401\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned shuffle 6\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 511\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 268\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 431\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 429\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 533\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 327\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 366\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 307\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 197\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 376\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 390\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 198\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 309\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 426\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 489\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 559\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 330\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 525\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 383\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 225\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 201\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 217\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 258\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 504\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 480\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 389\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_16_piece0 on 10.0.243.141:40637 in memory (size: 16.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_16_piece0 on algo-1:36955 in memory (size: 16.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 514\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 172\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 529\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 458\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 469\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 224\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 246\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 273\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 487\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 339\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 340\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 259\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 267\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 319\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 187\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 524\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 347\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 545\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 361\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 209\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 574\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 528\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 549\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 279\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 302\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 179\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 332\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_14_piece0 on 10.0.243.141:40637 in memory (size: 26.6 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_14_piece0 on algo-1:36955 in memory (size: 26.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 296\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 540\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 443\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 526\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 308\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 465\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 164\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 346\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 260\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 344\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 194\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 456\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 283\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Registering RDD 105 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Got job 12 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Final stage: ResultStage 20 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 19)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 19)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 19 (MapPartitionsRDD[105] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_9_piece0 on 10.0.243.141:40637 in memory (size: 16.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_22 stored as values in memory (estimated size 109.3 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_9_piece0 on algo-1:36955 in memory (size: 16.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_22_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_22_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Created broadcast 22 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 371\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 196\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 245\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_21_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 19 (MapPartitionsRDD[105] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Adding task set 19.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Starting task 0.0 in stage 19.0 (TID 19, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_21_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 298\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 428\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 466\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 363\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 491\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 214\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 516\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 359\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 357\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_10_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_10_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_22_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 523\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 342\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 292\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 388\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 468\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 310\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 477\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 463\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 200\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 492\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 169\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 410\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_12_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Removed broadcast_12_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 498\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 537\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 133\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 250\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 248\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 219\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 509\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 334\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 415\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 210\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 203\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 352\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 534\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 251\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 362\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 490\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 255\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 459\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 423\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 316\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 481\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 513\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 356\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 221\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 240\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 175\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 168\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 387\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 220\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 277\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 181\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 518\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 134\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 272\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 507\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 449\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 276\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 199\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 422\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 527\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 476\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 353\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 205\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 482\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 321\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 467\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 573\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 473\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 488\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 450\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 424\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  ContextCleaner:54 - Cleaned accumulator 568\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Finished task 0.0 in stage 19.0 (TID 19) in 106 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Removed TaskSet 19.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - ShuffleMapStage 19 (collect at AnalysisRunner.scala:313) finished in 0.122 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - waiting: Set(ResultStage 20)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting ResultStage 20 (MapPartitionsRDD[108] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_23 stored as values in memory (estimated size 55.0 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_23_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_23_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Created broadcast 23 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 20 (MapPartitionsRDD[108] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Adding task set 20.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Starting task 0.0 in stage 20.0 (TID 20, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_23_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 7 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Finished task 0.0 in stage 20.0 (TID 20) in 77 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Removed TaskSet 20.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - ResultStage 20 (collect at AnalysisRunner.scala:313) finished in 0.086 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Job 12 finished: collect at AnalysisRunner.scala:313, took 0.215344 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Registering RDD 115 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Got job 13 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Final stage: ResultStage 22 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 21)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 21)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 21 (MapPartitionsRDD[115] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_24 stored as values in memory (estimated size 63.5 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_24_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_24_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Created broadcast 24 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 21 (MapPartitionsRDD[115] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Adding task set 21.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Starting task 0.0 in stage 21.0 (TID 21, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_24_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Finished task 0.0 in stage 21.0 (TID 21) in 69 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Removed TaskSet 21.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - ShuffleMapStage 21 (countByKey at ColumnProfiler.scala:566) finished in 0.081 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - waiting: Set(ResultStage 22)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting ResultStage 22 (ShuffledRDD[116] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_25 stored as values in memory (estimated size 3.2 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  MemoryStore:54 - Block broadcast_25_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  BlockManagerInfo:54 - Added broadcast_25_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  SparkContext:54 - Created broadcast 25 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 22 (ShuffledRDD[116] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  YarnScheduler:54 - Adding task set 22.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:17 INFO  TaskSetManager:54 - Starting task 0.0 in stage 22.0 (TID 22, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_25_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 8 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 22.0 (TID 22) in 39 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 22.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ResultStage 22 (countByKey at ColumnProfiler.scala:566) finished in 0.047 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Job 13 finished: countByKey at ColumnProfiler.scala:566, took 0.133128 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Registering RDD 121 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Got job 14 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Final stage: ResultStage 24 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 23)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 23)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 23 (MapPartitionsRDD[121] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_26 stored as values in memory (estimated size 108.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_26_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_26_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 26 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 23 (MapPartitionsRDD[121] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 23.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 23.0 (TID 23, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_26_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 23.0 (TID 23) in 105 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 23.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ShuffleMapStage 23 (collect at AnalysisRunner.scala:313) finished in 0.115 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - waiting: Set(ResultStage 24)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ResultStage 24 (MapPartitionsRDD[124] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_27 stored as values in memory (estimated size 121.1 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_27_piece0 stored as bytes in memory (estimated size 40.8 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_27_piece0 in memory on 10.0.243.141:40637 (size: 40.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 27 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 24 (MapPartitionsRDD[124] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 24.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 24.0 (TID 24, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_27_piece0 in memory on algo-1:36955 (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 9 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 24.0 (TID 24) in 111 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 24.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ResultStage 24 (collect at AnalysisRunner.scala:313) finished in 0.123 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Job 14 finished: collect at AnalysisRunner.scala:313, took 0.246563 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  CodeGenerator:54 - Code generated in 8.457669 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Got job 15 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Final stage: ResultStage 25 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ResultStage 25 (MapPartitionsRDD[133] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_28 stored as values in memory (estimated size 69.1 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_28_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_28_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 28 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 25 (MapPartitionsRDD[133] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 25.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 25.0 (TID 25, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_28_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 25.0 (TID 25) in 67 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 25.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ResultStage 25 (treeReduce at KLLRunner.scala:107) finished in 0.076 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Job 15 finished: treeReduce at KLLRunner.scala:107, took 0.080537 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  CodeGenerator:54 - Code generated in 25.914723 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Registering RDD 138 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Got job 16 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Final stage: ResultStage 27 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 26)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 26)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 26 (MapPartitionsRDD[138] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_29 stored as values in memory (estimated size 109.3 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_29_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_29_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 29 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 26 (MapPartitionsRDD[138] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 26.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 26.0 (TID 26, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_29_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 26.0 (TID 26) in 89 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 26.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ShuffleMapStage 26 (collect at AnalysisRunner.scala:313) finished in 0.097 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - waiting: Set(ResultStage 27)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ResultStage 27 (MapPartitionsRDD[141] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_30 stored as values in memory (estimated size 55.0 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_30_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_30_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 30 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 27 (MapPartitionsRDD[141] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 27.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 27.0 (TID 27, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_30_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 10 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Finished task 0.0 in stage 27.0 (TID 27) in 28 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Removed TaskSet 27.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - ResultStage 27 (collect at AnalysisRunner.scala:313) finished in 0.039 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Job 16 finished: collect at AnalysisRunner.scala:313, took 0.143295 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Registering RDD 148 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Got job 17 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Final stage: ResultStage 29 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 28)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 28)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 28 (MapPartitionsRDD[148] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_31 stored as values in memory (estimated size 63.5 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  MemoryStore:54 - Block broadcast_31_piece0 stored as bytes in memory (estimated size 24.6 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_31_piece0 in memory on 10.0.243.141:40637 (size: 24.6 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  SparkContext:54 - Created broadcast 31 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 28 (MapPartitionsRDD[148] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  YarnScheduler:54 - Adding task set 28.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  TaskSetManager:54 - Starting task 0.0 in stage 28.0 (TID 28, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:18 INFO  BlockManagerInfo:54 - Added broadcast_31_piece0 in memory on algo-1:36955 (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 28.0 (TID 28) in 54 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 28.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ShuffleMapStage 28 (countByKey at ColumnProfiler.scala:566) finished in 0.078 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - waiting: Set(ResultStage 29)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ResultStage 29 (ShuffledRDD[149] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_32 stored as values in memory (estimated size 3.2 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_32_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_32_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 32 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 29 (ShuffledRDD[149] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 29.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 29.0 (TID 29, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_32_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 11 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 29.0 (TID 29) in 33 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 29.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ResultStage 29 (countByKey at ColumnProfiler.scala:566) finished in 0.041 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Job 17 finished: countByKey at ColumnProfiler.scala:566, took 0.127330 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Registering RDD 154 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Got job 18 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Final stage: ResultStage 31 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 30)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 30)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 30 (MapPartitionsRDD[154] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_33 stored as values in memory (estimated size 108.7 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_33_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_33_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 33 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 30 (MapPartitionsRDD[154] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 30.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 30.0 (TID 30, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_33_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 30.0 (TID 30) in 160 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 30.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ShuffleMapStage 30 (collect at AnalysisRunner.scala:313) finished in 0.173 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - waiting: Set(ResultStage 31)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ResultStage 31 (MapPartitionsRDD[157] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_34 stored as values in memory (estimated size 121.1 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_34_piece0 stored as bytes in memory (estimated size 40.9 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_34_piece0 in memory on 10.0.243.141:40637 (size: 40.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 34 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 31 (MapPartitionsRDD[157] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 31.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 31.0 (TID 31, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_34_piece0 in memory on algo-1:36955 (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 12 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 31.0 (TID 31) in 123 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 31.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ResultStage 31 (collect at AnalysisRunner.scala:313) finished in 0.138 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Job 18 finished: collect at AnalysisRunner.scala:313, took 0.318609 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Got job 19 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Final stage: ResultStage 32 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ResultStage 32 (MapPartitionsRDD[166] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_35 stored as values in memory (estimated size 69.1 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_35_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_35_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 35 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 32 (MapPartitionsRDD[166] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 32.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 32.0 (TID 32, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_35_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 32.0 (TID 32) in 50 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 32.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ResultStage 32 (treeReduce at KLLRunner.scala:107) finished in 0.059 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Job 19 finished: treeReduce at KLLRunner.scala:107, took 0.061818 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Registering RDD 171 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Got job 20 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Final stage: ResultStage 34 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 33)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 33)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 33 (MapPartitionsRDD[171] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_36 stored as values in memory (estimated size 109.3 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_36_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_36_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 36 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 33 (MapPartitionsRDD[171] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 33.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 33.0 (TID 33, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_36_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 33.0 (TID 33) in 27 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 33.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ShuffleMapStage 33 (collect at AnalysisRunner.scala:313) finished in 0.038 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - waiting: Set(ResultStage 34)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting ResultStage 34 (MapPartitionsRDD[174] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_37 stored as values in memory (estimated size 55.0 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 904\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MemoryStore:54 - Block broadcast_37_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 683\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 662\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 723\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_37_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 654\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 946\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 862\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 825\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 851\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 762\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  SparkContext:54 - Created broadcast 37 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 631\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 34 (MapPartitionsRDD[174] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Adding task set 34.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 832\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 622\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 660\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 647\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 585\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 848\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 928\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 670\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 995\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 922\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 745\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 784\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 847\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 957\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 953\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 658\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Starting task 0.0 in stage 34.0 (TID 34, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned shuffle 10\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 551\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 819\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 812\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 766\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 699\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 798\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 931\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 882\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 818\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_30_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_30_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 973\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 967\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 983\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 610\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 839\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 618\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 729\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 903\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 858\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 854\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 890\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 718\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 835\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 706\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 672\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 792\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 645\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 895\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 684\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 616\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 656\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 767\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 937\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 708\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 829\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Added broadcast_37_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_26_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_26_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 881\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 761\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 997\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 877\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 883\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 644\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 698\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 906\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 743\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 857\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 639\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 958\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 632\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 665\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 988\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 775\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 582\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 599\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 608\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 751\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 868\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 893\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 663\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 892\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 613\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 859\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned shuffle 8\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_31_piece0 on 10.0.243.141:40637 in memory (size: 24.6 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_31_piece0 on algo-1:36955 in memory (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 13 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 685\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 939\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 954\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 700\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 593\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 785\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 827\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 697\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 722\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 707\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 850\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 730\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 715\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 752\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 682\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 981\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 705\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 846\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 804\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 891\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 586\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 778\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 748\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 800\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 809\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 834\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 942\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 865\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 836\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 675\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 726\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 671\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 794\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 869\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 744\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 702\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 724\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 753\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 588\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 880\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 695\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 977\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 583\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 713\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  TaskSetManager:54 - Finished task 0.0 in stage 34.0 (TID 34) in 56 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  YarnScheduler:54 - Removed TaskSet 34.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_29_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - ResultStage 34 (collect at AnalysisRunner.scala:313) finished in 0.098 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  DAGScheduler:54 - Job 20 finished: collect at AnalysisRunner.scala:313, took 0.140235 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_29_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 951\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 984\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 771\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 634\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 789\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 609\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 787\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 912\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 764\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 870\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 673\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 964\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 596\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 650\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 636\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 691\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 701\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 806\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 911\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 934\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 747\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 628\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 873\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 741\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 935\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 986\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 661\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 779\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 774\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 758\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 797\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 742\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 813\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 972\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 602\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 605\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 710\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 959\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_35_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_35_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 617\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 807\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 927\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 941\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_28_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_28_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 765\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 750\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 755\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 842\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 994\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 659\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 746\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 614\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 817\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned shuffle 12\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 649\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 626\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 976\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 674\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 693\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 878\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 584\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 900\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 945\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 611\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 947\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 615\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 975\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 760\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 740\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 781\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 926\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 950\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 913\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 720\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 757\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 589\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 763\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 759\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 996\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 703\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 902\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 598\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 587\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 689\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 980\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 795\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 918\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 929\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 828\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 664\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 874\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 802\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 830\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 855\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 630\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 790\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_25_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  BlockManagerInfo:54 - Removed broadcast_25_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 721\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 894\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 667\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 694\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 681\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 840\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 638\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 657\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 949\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 993\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 591\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 776\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 788\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 965\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 907\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 861\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 791\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 956\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 756\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 952\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 849\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:19 INFO  ContextCleaner:54 - Cleaned accumulator 655\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_23_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_23_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 768\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 553\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 590\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 619\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 633\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 930\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 717\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 680\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 808\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 782\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 642\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 814\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 627\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 879\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 979\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 920\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 889\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 736\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 833\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 822\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 709\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 940\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 932\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 896\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 604\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 810\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 982\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 716\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 886\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 968\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned shuffle 11\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 823\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 985\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 837\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 783\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned shuffle 9\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 820\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 749\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 811\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 769\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 648\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 653\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 961\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 637\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 955\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 625\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 712\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 974\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_24_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_24_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 799\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 690\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 629\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 738\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 884\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 777\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 831\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 943\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 948\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 915\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 897\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 594\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 732\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_27_piece0 on 10.0.243.141:40637 in memory (size: 40.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_27_piece0 on algo-1:36955 in memory (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 914\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 987\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 872\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 989\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 595\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 909\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 921\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 635\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 801\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 908\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 871\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 725\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 652\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 845\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 692\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_32_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_32_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 966\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 841\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 640\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 824\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 866\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 936\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 898\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 816\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 960\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 860\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 580\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 925\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_33_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_33_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 641\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Registering RDD 181 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Got job 21 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Final stage: ResultStage 36 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 35)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 35)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_34_piece0 on 10.0.243.141:40637 in memory (size: 40.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 35 (MapPartitionsRDD[181] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_34_piece0 on algo-1:36955 in memory (size: 40.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 933\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 815\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned shuffle 7\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 905\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 901\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 962\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 678\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 803\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 899\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 923\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 919\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 853\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 944\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 805\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 677\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 992\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 826\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 864\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 770\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_38 stored as values in memory (estimated size 63.5 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_38_piece0 stored as bytes in memory (estimated size 24.6 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 612\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 581\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 731\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 856\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 687\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 714\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 719\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 669\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 875\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 793\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 704\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 867\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 739\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 773\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 696\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 623\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 821\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 601\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 624\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 711\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 646\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 651\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 938\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 863\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 666\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 728\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 991\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 679\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 990\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 592\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_38_piece0 in memory on 10.0.243.141:40637 (size: 24.6 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_22_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 38 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 35 (MapPartitionsRDD[181] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 35.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Removed broadcast_22_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 35.0 (TID 35, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 727\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 876\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 843\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 643\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 733\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 552\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 838\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 621\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 888\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 734\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 754\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 917\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 844\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 597\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 607\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 916\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 688\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 780\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 603\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 978\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 885\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 735\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 737\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 772\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 676\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 786\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 963\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 924\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 887\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 686\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 796\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 910\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 852\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 600\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 620\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 606\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  ContextCleaner:54 - Cleaned accumulator 668\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_38_piece0 in memory on algo-1:36955 (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 35.0 (TID 35) in 56 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 35.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ShuffleMapStage 35 (countByKey at ColumnProfiler.scala:566) finished in 0.075 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - waiting: Set(ResultStage 36)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ResultStage 36 (ShuffledRDD[182] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_39 stored as values in memory (estimated size 3.2 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_39_piece0 stored as bytes in memory (estimated size 1925.0 B, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_39_piece0 in memory on 10.0.243.141:40637 (size: 1925.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 39 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 36 (ShuffledRDD[182] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 36.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 36.0 (TID 36, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_39_piece0 in memory on algo-1:36955 (size: 1925.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 14 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 36.0 (TID 36) in 21 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 36.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ResultStage 36 (countByKey at ColumnProfiler.scala:566) finished in 0.038 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Job 21 finished: countByKey at ColumnProfiler.scala:566, took 0.119215 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Registering RDD 187 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Got job 22 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Final stage: ResultStage 38 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 37)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 37)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 37 (MapPartitionsRDD[187] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_40 stored as values in memory (estimated size 108.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_40_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_40_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 40 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 37 (MapPartitionsRDD[187] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 37.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 37.0 (TID 37, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_40_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 37.0 (TID 37) in 136 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 37.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ShuffleMapStage 37 (collect at AnalysisRunner.scala:313) finished in 0.144 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - waiting: Set(ResultStage 38)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ResultStage 38 (MapPartitionsRDD[190] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_41 stored as values in memory (estimated size 121.1 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_41_piece0 stored as bytes in memory (estimated size 40.7 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_41_piece0 in memory on 10.0.243.141:40637 (size: 40.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 41 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 38 (MapPartitionsRDD[190] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 38.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 38.0 (TID 38, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_41_piece0 in memory on algo-1:36955 (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 15 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 38.0 (TID 38) in 103 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 38.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ResultStage 38 (collect at AnalysisRunner.scala:313) finished in 0.125 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Job 22 finished: collect at AnalysisRunner.scala:313, took 0.272151 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Got job 23 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Final stage: ResultStage 39 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ResultStage 39 (MapPartitionsRDD[199] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_42 stored as values in memory (estimated size 69.1 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_42_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_42_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 42 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 39 (MapPartitionsRDD[199] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 39.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 39.0 (TID 39, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_42_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 39.0 (TID 39) in 54 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 39.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ResultStage 39 (treeReduce at KLLRunner.scala:107) finished in 0.063 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Job 23 finished: treeReduce at KLLRunner.scala:107, took 0.065866 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Registering RDD 204 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Got job 24 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Final stage: ResultStage 41 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 40)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 40)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 40 (MapPartitionsRDD[204] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_43 stored as values in memory (estimated size 109.3 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_43_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_43_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 43 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 40 (MapPartitionsRDD[204] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 40.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 40.0 (TID 40, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_43_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 40.0 (TID 40) in 29 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 40.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ShuffleMapStage 40 (collect at AnalysisRunner.scala:313) finished in 0.037 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - waiting: Set(ResultStage 41)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ResultStage 41 (MapPartitionsRDD[207] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_44 stored as values in memory (estimated size 55.0 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_44_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_44_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 44 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 41 (MapPartitionsRDD[207] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 41.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 41.0 (TID 41, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_44_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 16 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 41.0 (TID 41) in 17 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 41.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - ResultStage 41 (collect at AnalysisRunner.scala:313) finished in 0.025 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Job 24 finished: collect at AnalysisRunner.scala:313, took 0.072027 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Registering RDD 214 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Got job 25 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Final stage: ResultStage 43 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 42)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 42)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 42 (MapPartitionsRDD[214] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_45 stored as values in memory (estimated size 63.5 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  MemoryStore:54 - Block broadcast_45_piece0 stored as bytes in memory (estimated size 24.6 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_45_piece0 in memory on 10.0.243.141:40637 (size: 24.6 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  SparkContext:54 - Created broadcast 45 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 42 (MapPartitionsRDD[214] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Adding task set 42.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Starting task 0.0 in stage 42.0 (TID 42, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  BlockManagerInfo:54 - Added broadcast_45_piece0 in memory on algo-1:36955 (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  TaskSetManager:54 - Finished task 0.0 in stage 42.0 (TID 42) in 57 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:20 INFO  YarnScheduler:54 - Removed TaskSet 42.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ShuffleMapStage 42 (countByKey at ColumnProfiler.scala:566) finished in 0.080 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - waiting: Set(ResultStage 43)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ResultStage 43 (ShuffledRDD[215] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_46 stored as values in memory (estimated size 3.2 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_46_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_46_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 46 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 43 (ShuffledRDD[215] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 43.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 43.0 (TID 43, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_46_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 17 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 43.0 (TID 43) in 43 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 43.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ResultStage 43 (countByKey at ColumnProfiler.scala:566) finished in 0.051 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Job 25 finished: countByKey at ColumnProfiler.scala:566, took 0.136240 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Registering RDD 220 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Got job 26 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Final stage: ResultStage 45 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 44)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 44)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 44 (MapPartitionsRDD[220] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_47 stored as values in memory (estimated size 108.7 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_47_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_47_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 47 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 44 (MapPartitionsRDD[220] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 44.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 44.0 (TID 44, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_47_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 44.0 (TID 44) in 108 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 44.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ShuffleMapStage 44 (collect at AnalysisRunner.scala:313) finished in 0.116 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - waiting: Set(ResultStage 45)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ResultStage 45 (MapPartitionsRDD[223] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_48 stored as values in memory (estimated size 121.1 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_48_piece0 stored as bytes in memory (estimated size 40.7 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_48_piece0 in memory on 10.0.243.141:40637 (size: 40.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 48 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 45 (MapPartitionsRDD[223] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 45.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 45.0 (TID 45, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_48_piece0 in memory on algo-1:36955 (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 18 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 45.0 (TID 45) in 114 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 45.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ResultStage 45 (collect at AnalysisRunner.scala:313) finished in 0.123 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Job 26 finished: collect at AnalysisRunner.scala:313, took 0.243269 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Got job 27 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Final stage: ResultStage 46 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ResultStage 46 (MapPartitionsRDD[232] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_49 stored as values in memory (estimated size 69.1 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_49_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_49_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 49 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 46 (MapPartitionsRDD[232] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 46.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 46.0 (TID 46, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_49_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 46.0 (TID 46) in 40 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 46.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ResultStage 46 (treeReduce at KLLRunner.scala:107) finished in 0.048 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Job 27 finished: treeReduce at KLLRunner.scala:107, took 0.050415 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Registering RDD 237 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Got job 28 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Final stage: ResultStage 48 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 47)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 47)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 47 (MapPartitionsRDD[237] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_50 stored as values in memory (estimated size 109.3 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_50_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_50_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 50 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 47 (MapPartitionsRDD[237] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 47.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 47.0 (TID 47, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_50_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 47.0 (TID 47) in 39 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 47.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ShuffleMapStage 47 (collect at AnalysisRunner.scala:313) finished in 0.049 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - waiting: Set(ResultStage 48)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ResultStage 48 (MapPartitionsRDD[240] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_51 stored as values in memory (estimated size 55.0 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_51_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_51_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 51 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 48 (MapPartitionsRDD[240] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 48.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 48.0 (TID 48, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_51_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 19 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 48.0 (TID 48) in 30 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 48.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ResultStage 48 (collect at AnalysisRunner.scala:313) finished in 0.037 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Job 28 finished: collect at AnalysisRunner.scala:313, took 0.092628 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Registering RDD 247 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Got job 29 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Final stage: ResultStage 50 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 49)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 49)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 49 (MapPartitionsRDD[247] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_52 stored as values in memory (estimated size 63.5 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_52_piece0 stored as bytes in memory (estimated size 24.6 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_52_piece0 in memory on 10.0.243.141:40637 (size: 24.6 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 52 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 49 (MapPartitionsRDD[247] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 49.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 49.0 (TID 49, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_52_piece0 in memory on algo-1:36955 (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 49.0 (TID 49) in 39 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 49.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ShuffleMapStage 49 (countByKey at ColumnProfiler.scala:566) finished in 0.050 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - waiting: Set(ResultStage 50)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ResultStage 50 (ShuffledRDD[248] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_53 stored as values in memory (estimated size 3.2 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_53_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_53_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 53 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 50 (ShuffledRDD[248] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 50.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 50.0 (TID 50, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_53_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 20 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Finished task 0.0 in stage 50.0 (TID 50) in 25 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Removed TaskSet 50.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - ResultStage 50 (countByKey at ColumnProfiler.scala:566) finished in 0.032 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Job 29 finished: countByKey at ColumnProfiler.scala:566, took 0.086187 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Registering RDD 253 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Got job 30 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Final stage: ResultStage 52 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 51)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 51)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 51 (MapPartitionsRDD[253] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_54 stored as values in memory (estimated size 108.7 KB, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  MemoryStore:54 - Block broadcast_54_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_54_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  SparkContext:54 - Created broadcast 54 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 51 (MapPartitionsRDD[253] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  YarnScheduler:54 - Adding task set 51.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  TaskSetManager:54 - Starting task 0.0 in stage 51.0 (TID 51, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:21 INFO  BlockManagerInfo:54 - Added broadcast_54_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 51.0 (TID 51) in 79 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 51.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ShuffleMapStage 51 (collect at AnalysisRunner.scala:313) finished in 0.088 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - waiting: Set(ResultStage 52)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ResultStage 52 (MapPartitionsRDD[256] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_55 stored as values in memory (estimated size 121.1 KB, free 1456.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_55_piece0 stored as bytes in memory (estimated size 40.7 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_55_piece0 in memory on 10.0.243.141:40637 (size: 40.7 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 55 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 52 (MapPartitionsRDD[256] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 52.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 52.0 (TID 52, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_55_piece0 in memory on algo-1:36955 (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 21 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 52.0 (TID 52) in 69 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 52.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ResultStage 52 (collect at AnalysisRunner.scala:313) finished in 0.078 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Job 30 finished: collect at AnalysisRunner.scala:313, took 0.168964 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Got job 31 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Final stage: ResultStage 53 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ResultStage 53 (MapPartitionsRDD[265] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_56 stored as values in memory (estimated size 69.1 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_56_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_56_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 56 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 53 (MapPartitionsRDD[265] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 53.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 53.0 (TID 53, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_56_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 53.0 (TID 53) in 35 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 53.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ResultStage 53 (treeReduce at KLLRunner.scala:107) finished in 0.043 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Job 31 finished: treeReduce at KLLRunner.scala:107, took 0.045492 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1518\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1015\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1448\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1135\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1600\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1059\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1319\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1200\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1387\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1572\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1099\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1292\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1183\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1520\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1538\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1150\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1009\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1023\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1557\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1044\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1047\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1123\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1576\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1213\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 19\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1195\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1010\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1579\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1288\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1193\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1234\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1441\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1056\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1277\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1338\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1188\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1563\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1066\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1624\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1559\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1294\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1271\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1330\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1392\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1444\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 999\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1418\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1521\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1342\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1405\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1586\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1526\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1181\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1415\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_51_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_51_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1309\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1347\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1394\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1030\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1062\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 14\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1004\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1427\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1256\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1174\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1032\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1328\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_55_piece0 on algo-1:36955 in memory (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_55_piece0 on 10.0.243.141:40637 in memory (size: 40.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1255\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1038\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1550\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1505\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1568\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1435\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1482\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1384\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1419\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1307\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_50_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_50_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1479\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1251\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1591\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1139\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1248\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1404\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1311\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1602\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1146\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1450\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1565\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1472\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1352\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1049\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1617\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1470\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1372\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1580\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1057\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1245\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1264\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1445\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1547\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1609\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1525\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1555\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1217\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1276\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1036\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1140\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1215\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1549\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1569\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1013\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1001\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1006\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1163\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 998\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1322\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1430\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1109\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1301\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1303\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1573\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1060\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1191\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1432\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1098\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1312\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1161\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1423\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1608\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1265\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1316\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1543\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1439\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_52_piece0 on 10.0.243.141:40637 in memory (size: 24.6 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_52_piece0 on algo-1:36955 in memory (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1523\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1345\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1250\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1048\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1507\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1567\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1460\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1475\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1131\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1454\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1083\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1339\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1074\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1340\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1376\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1615\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1485\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1408\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1037\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1148\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1239\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1022\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1402\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1244\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1502\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1326\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1052\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1287\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1305\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1084\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1551\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1147\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1238\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1317\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1362\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1556\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1203\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1621\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1410\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1137\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1546\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1017\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_37_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_37_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1152\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1516\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1034\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1252\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1299\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1053\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 970\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1522\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1552\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_48_piece0 on 10.0.243.141:40637 in memory (size: 40.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Registering RDD 270 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Got job 32 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Final stage: ResultStage 55 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 54)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 54)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 54 (MapPartitionsRDD[270] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_48_piece0 on algo-1:36955 in memory (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1144\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1469\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1499\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1063\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1329\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1079\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1412\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1461\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1468\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_57 stored as values in memory (estimated size 109.3 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 16\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1424\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1281\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1604\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1377\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1132\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1016\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1020\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1508\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1170\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1101\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1149\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1289\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1184\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1446\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1588\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1122\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1235\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1325\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1085\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1205\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1258\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1466\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1142\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1211\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1332\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1524\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1440\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1263\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1018\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1041\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1021\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1371\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1324\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1587\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1442\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1220\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1293\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1321\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1388\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1357\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1151\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1257\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1554\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1530\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1354\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1379\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_57_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1061\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1407\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1229\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_57_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 57 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 54 (MapPartitionsRDD[270] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_44_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 54.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 54.0 (TID 54, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_44_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1064\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1333\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1027\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1571\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1214\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1436\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1449\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1029\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1233\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1267\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1426\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1533\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1489\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1583\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1198\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1254\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1375\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1284\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1561\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1582\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1327\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1231\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1031\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1463\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1055\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1607\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1241\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1606\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1033\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1089\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1237\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1104\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1216\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1503\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1042\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1112\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1336\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1290\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1476\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1024\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1353\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1364\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1154\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1087\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1261\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1168\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1199\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1501\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1105\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1113\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1545\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1611\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1003\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1519\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1363\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1622\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1196\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1395\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1019\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1046\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1495\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1090\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_57_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_39_piece0 on 10.0.243.141:40637 in memory (size: 1925.0 B, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_39_piece0 on algo-1:36955 in memory (size: 1925.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1268\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1417\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1447\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1478\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1548\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1176\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1242\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1486\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1225\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1471\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1116\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1067\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_49_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_49_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 54.0 (TID 54) in 22 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1374\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1401\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1179\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 54.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1286\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ShuffleMapStage 54 (collect at AnalysisRunner.scala:313) finished in 0.034 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - waiting: Set(ResultStage 55)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1086\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1539\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1260\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1334\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ResultStage 55 (MapPartitionsRDD[273] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_58 stored as values in memory (estimated size 55.0 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1457\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1035\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1136\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1411\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1529\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1028\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1247\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1127\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_43_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_58_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_43_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_58_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 58 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1262\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1040\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1097\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1350\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 55 (MapPartitionsRDD[273] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 55.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1462\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1323\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 55.0 (TID 55, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_53_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_53_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1577\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1594\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1172\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1130\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 18\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1496\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1065\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1467\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1393\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1182\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1459\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1120\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1433\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1189\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1341\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1542\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1157\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1108\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1259\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1073\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1106\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1483\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_40_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_40_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1358\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1453\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1397\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1246\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1595\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1201\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1318\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1400\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1492\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1221\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1143\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1389\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1298\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1490\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1391\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1527\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1273\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1531\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_58_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1618\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1473\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1406\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_41_piece0 on 10.0.243.141:40637 in memory (size: 40.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_41_piece0 on algo-1:36955 in memory (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1002\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1280\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1509\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1081\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1614\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1360\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1382\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1194\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1474\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_38_piece0 on 10.0.243.141:40637 in memory (size: 24.6 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_38_piece0 on algo-1:36955 in memory (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1110\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1190\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1437\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1465\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1111\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1428\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1536\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1117\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1403\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 969\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1180\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1612\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1094\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1620\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1514\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1134\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1212\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1619\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1162\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_42_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_42_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1541\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1425\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1272\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1088\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1095\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1266\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1308\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1369\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1008\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1452\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1218\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1578\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1278\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1399\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1351\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1560\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1160\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1511\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1420\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1370\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1383\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1207\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1346\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1564\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1078\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1356\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1185\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1337\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1585\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1431\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1249\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1359\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1493\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1129\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1153\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1232\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1014\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1535\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1243\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1575\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1378\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1093\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1124\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1416\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1050\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1413\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1007\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1314\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1584\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1302\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1125\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1283\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1295\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 21\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1138\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1269\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1361\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 22 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1421\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1562\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1051\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1282\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1344\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1167\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1114\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1491\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1025\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1082\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1386\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1455\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1208\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1603\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1043\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1367\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1500\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1012\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1228\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1291\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1192\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1274\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1515\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1005\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1300\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1398\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1443\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1230\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1121\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1380\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1592\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1076\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1165\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1365\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1348\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1075\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1133\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1164\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1126\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1159\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1574\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1512\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1275\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1096\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1011\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1158\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 17\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1537\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1119\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1335\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1206\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1296\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1605\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 971\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1306\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_47_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_47_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 55.0 (TID 55) in 55 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 55.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ResultStage 55 (collect at AnalysisRunner.scala:313) finished in 0.064 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1381\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1270\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1349\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1429\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1209\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_45_piece0 on 10.0.243.141:40637 in memory (size: 24.6 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Job 32 finished: collect at AnalysisRunner.scala:313, took 0.103931 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Registering RDD 280 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Got job 33 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Final stage: ResultStage 57 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 56)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 56)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 56 (MapPartitionsRDD[280] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_59 stored as values in memory (estimated size 63.5 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_59_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_59_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 59 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 56 (MapPartitionsRDD[280] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 56.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 56.0 (TID 56, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_45_piece0 on algo-1:36955 in memory (size: 24.6 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1623\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_46_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_59_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_46_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1601\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1155\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1480\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_36_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_36_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1175\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1517\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1534\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1219\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1171\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1593\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1227\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1077\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1222\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1504\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1558\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 20\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1202\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1226\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1498\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1570\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1279\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1451\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1422\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1510\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1210\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1497\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 15\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1315\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1045\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1177\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1494\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1197\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1456\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1187\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1166\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1072\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1355\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1343\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1373\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1581\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned shuffle 13\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1223\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1589\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1438\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1224\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1169\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1396\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1058\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1434\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1071\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1070\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1100\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1236\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1297\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_54_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 56.0 (TID 56) in 180 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 56.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ShuffleMapStage 56 (countByKey at ColumnProfiler.scala:566) finished in 0.191 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - waiting: Set(ResultStage 57)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_54_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ResultStage 57 (ShuffledRDD[281] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_60 stored as values in memory (estimated size 3.2 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_60_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1118\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1464\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1173\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_60_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1304\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1506\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1544\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1528\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 60 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_56_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 57 (ShuffledRDD[281] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 57.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 57.0 (TID 57, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Removed broadcast_56_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1092\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1026\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1385\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1054\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1458\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1532\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1590\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1540\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1613\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1128\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1331\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1599\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1320\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1414\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1156\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1484\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1186\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1204\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1310\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1102\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1069\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1409\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1513\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1285\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1080\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1368\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1610\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1553\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1141\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1253\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1178\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1091\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1103\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1145\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1115\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1240\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1488\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1616\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1366\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1390\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1481\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1477\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  ContextCleaner:54 - Cleaned accumulator 1487\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_60_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 23 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 57.0 (TID 57) in 20 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 57.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ResultStage 57 (countByKey at ColumnProfiler.scala:566) finished in 0.029 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Job 33 finished: countByKey at ColumnProfiler.scala:566, took 0.224489 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Registering RDD 286 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Got job 34 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Final stage: ResultStage 59 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 58)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 58)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 58 (MapPartitionsRDD[286] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_61 stored as values in memory (estimated size 108.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_61_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_61_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 61 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 58 (MapPartitionsRDD[286] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 58.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 58.0 (TID 58, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_61_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Finished task 0.0 in stage 58.0 (TID 58) in 79 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Removed TaskSet 58.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - ShuffleMapStage 58 (collect at AnalysisRunner.scala:313) finished in 0.086 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - waiting: Set(ResultStage 59)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting ResultStage 59 (MapPartitionsRDD[289] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_62 stored as values in memory (estimated size 121.1 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  MemoryStore:54 - Block broadcast_62_piece0 stored as bytes in memory (estimated size 40.8 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  BlockManagerInfo:54 - Added broadcast_62_piece0 in memory on 10.0.243.141:40637 (size: 40.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  SparkContext:54 - Created broadcast 62 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 59 (MapPartitionsRDD[289] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  YarnScheduler:54 - Adding task set 59.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:22 INFO  TaskSetManager:54 - Starting task 0.0 in stage 59.0 (TID 59, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_62_piece0 in memory on algo-1:36955 (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 24 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 59.0 (TID 59) in 73 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 59.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 59 (collect at AnalysisRunner.scala:313) finished in 0.081 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 34 finished: collect at AnalysisRunner.scala:313, took 0.171379 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 35 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 60 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 60 (MapPartitionsRDD[298] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_63 stored as values in memory (estimated size 69.1 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_63_piece0 stored as bytes in memory (estimated size 26.3 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_63_piece0 in memory on 10.0.243.141:40637 (size: 26.3 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 63 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 60 (MapPartitionsRDD[298] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 60.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 60.0 (TID 60, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_63_piece0 in memory on algo-1:36955 (size: 26.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 60.0 (TID 60) in 42 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 60.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 60 (treeReduce at KLLRunner.scala:107) finished in 0.048 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 35 finished: treeReduce at KLLRunner.scala:107, took 0.049860 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Registering RDD 303 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 36 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 62 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 61)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 61)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 61 (MapPartitionsRDD[303] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_64 stored as values in memory (estimated size 109.3 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_64_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1457.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_64_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 64 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 61 (MapPartitionsRDD[303] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 61.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 61.0 (TID 61, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_64_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 61.0 (TID 61) in 17 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 61.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ShuffleMapStage 61 (collect at AnalysisRunner.scala:313) finished in 0.023 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - waiting: Set(ResultStage 62)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 62 (MapPartitionsRDD[306] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_65 stored as values in memory (estimated size 55.0 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_65_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_65_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 65 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 62 (MapPartitionsRDD[306] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 62.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 62.0 (TID 62, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_65_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 25 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 62.0 (TID 62) in 14 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 62.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 62 (collect at AnalysisRunner.scala:313) finished in 0.019 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 36 finished: collect at AnalysisRunner.scala:313, took 0.048489 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Registering RDD 313 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 37 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 64 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 63)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 63)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 63 (MapPartitionsRDD[313] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_66 stored as values in memory (estimated size 63.5 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_66_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_66_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 66 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 63 (MapPartitionsRDD[313] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 63.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 63.0 (TID 63, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_66_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 63.0 (TID 63) in 30 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 63.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ShuffleMapStage 63 (countByKey at ColumnProfiler.scala:566) finished in 0.040 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - waiting: Set(ResultStage 64)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 64 (ShuffledRDD[314] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_67 stored as values in memory (estimated size 3.2 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_67_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_67_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 67 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 64 (ShuffledRDD[314] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 64.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 64.0 (TID 64, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_67_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 26 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 64.0 (TID 64) in 21 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 64.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 64 (countByKey at ColumnProfiler.scala:566) finished in 0.034 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 37 finished: countByKey at ColumnProfiler.scala:566, took 0.077089 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Registering RDD 319 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 38 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 66 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 65)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 65)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 65 (MapPartitionsRDD[319] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_68 stored as values in memory (estimated size 108.7 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_68_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_68_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 68 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 65 (MapPartitionsRDD[319] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 65.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 65.0 (TID 65, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_68_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 65.0 (TID 65) in 75 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 65.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ShuffleMapStage 65 (collect at AnalysisRunner.scala:313) finished in 0.081 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - waiting: Set(ResultStage 66)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 66 (MapPartitionsRDD[322] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_69 stored as values in memory (estimated size 121.1 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_69_piece0 stored as bytes in memory (estimated size 40.7 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_69_piece0 in memory on 10.0.243.141:40637 (size: 40.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 69 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 66 (MapPartitionsRDD[322] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 66.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 66.0 (TID 66, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_69_piece0 in memory on algo-1:36955 (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 27 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 66.0 (TID 66) in 66 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 66.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 66 (collect at AnalysisRunner.scala:313) finished in 0.074 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 38 finished: collect at AnalysisRunner.scala:313, took 0.160221 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 39 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 67 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 67 (MapPartitionsRDD[331] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_70 stored as values in memory (estimated size 69.1 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_70_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_70_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 70 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 67 (MapPartitionsRDD[331] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 67.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 67.0 (TID 67, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_70_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 67.0 (TID 67) in 38 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 67.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 67 (treeReduce at KLLRunner.scala:107) finished in 0.046 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 39 finished: treeReduce at KLLRunner.scala:107, took 0.048372 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Registering RDD 336 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 40 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 69 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 68)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 68)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 68 (MapPartitionsRDD[336] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_71 stored as values in memory (estimated size 109.3 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_71_piece0 stored as bytes in memory (estimated size 37.7 KB, free 1456.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_71_piece0 in memory on 10.0.243.141:40637 (size: 37.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 71 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 68 (MapPartitionsRDD[336] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 68.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 68.0 (TID 68, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_71_piece0 in memory on algo-1:36955 (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 68.0 (TID 68) in 19 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 68.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ShuffleMapStage 68 (collect at AnalysisRunner.scala:313) finished in 0.028 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - waiting: Set(ResultStage 69)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 69 (MapPartitionsRDD[339] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_72 stored as values in memory (estimated size 55.0 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_72_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_72_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 72 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 69 (MapPartitionsRDD[339] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 69.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 69.0 (TID 69, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_72_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 28 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 69.0 (TID 69) in 15 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 69.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ResultStage 69 (collect at AnalysisRunner.scala:313) finished in 0.020 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Job 40 finished: collect at AnalysisRunner.scala:313, took 0.051252 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Registering RDD 346 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Got job 41 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Final stage: ResultStage 71 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 70)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 70)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 70 (MapPartitionsRDD[346] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_73 stored as values in memory (estimated size 63.5 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_73_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_73_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 73 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 70 (MapPartitionsRDD[346] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 70.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 70.0 (TID 70, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_73_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Finished task 0.0 in stage 70.0 (TID 70) in 23 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Removed TaskSet 70.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - ShuffleMapStage 70 (countByKey at ColumnProfiler.scala:566) finished in 0.029 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - waiting: Set(ResultStage 71)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting ResultStage 71 (ShuffledRDD[347] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_74 stored as values in memory (estimated size 3.2 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MemoryStore:54 - Block broadcast_74_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_74_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  SparkContext:54 - Created broadcast 74 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 71 (ShuffledRDD[347] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  YarnScheduler:54 - Adding task set 71.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  TaskSetManager:54 - Starting task 0.0 in stage 71.0 (TID 71, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  BlockManagerInfo:54 - Added broadcast_74_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:23 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 29 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 71.0 (TID 71) in 14 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 71.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ResultStage 71 (countByKey at ColumnProfiler.scala:566) finished in 0.020 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Job 41 finished: countByKey at ColumnProfiler.scala:566, took 0.052760 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Registering RDD 352 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Got job 42 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Final stage: ResultStage 73 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 72)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 72)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 72 (MapPartitionsRDD[352] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_75 stored as values in memory (estimated size 108.7 KB, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_75_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_75_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 75 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 72 (MapPartitionsRDD[352] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 72.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 72.0 (TID 72, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_75_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 72.0 (TID 72) in 83 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 72.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ShuffleMapStage 72 (collect at AnalysisRunner.scala:313) finished in 0.092 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - waiting: Set(ResultStage 73)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ResultStage 73 (MapPartitionsRDD[355] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_76 stored as values in memory (estimated size 121.1 KB, free 1456.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_76_piece0 stored as bytes in memory (estimated size 40.8 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_76_piece0 in memory on 10.0.243.141:40637 (size: 40.8 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 76 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 73 (MapPartitionsRDD[355] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 73.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 73.0 (TID 73, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_76_piece0 in memory on algo-1:36955 (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 30 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 73.0 (TID 73) in 80 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 73.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ResultStage 73 (collect at AnalysisRunner.scala:313) finished in 0.088 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Job 42 finished: collect at AnalysisRunner.scala:313, took 0.183114 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Got job 43 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Final stage: ResultStage 74 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ResultStage 74 (MapPartitionsRDD[364] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_77 stored as values in memory (estimated size 69.1 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_77_piece0 stored as bytes in memory (estimated size 26.3 KB, free 1456.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_77_piece0 in memory on 10.0.243.141:40637 (size: 26.3 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1876\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2077\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1734\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 77 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1865\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2026\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1781\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1758\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2159\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 74 (MapPartitionsRDD[364] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2053\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1766\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1853\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2156\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1658\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 74.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1935\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 74.0 (TID 74, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_65_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_65_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2169\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1874\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1898\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2119\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1914\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2093\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1878\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1662\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2023\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1815\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1842\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1628\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1934\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1687\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1833\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2060\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1723\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1818\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2059\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1886\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2099\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2178\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1995\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2035\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_66_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_66_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1948\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1880\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1978\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2166\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1689\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2075\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2067\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1816\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1922\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1715\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_77_piece0 in memory on algo-1:36955 (size: 26.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1772\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_57_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_57_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1819\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1916\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1909\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1960\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1972\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2070\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2213\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2145\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1954\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2103\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1924\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1700\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1762\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1596\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2124\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2051\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1967\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1811\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1784\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2058\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1872\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1641\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1764\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1986\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2025\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2164\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2158\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2161\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1724\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1976\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1778\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1820\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1638\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1864\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1598\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2140\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1962\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1671\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2137\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1854\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_76_piece0 on 10.0.243.141:40637 in memory (size: 40.8 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_76_piece0 on algo-1:36955 in memory (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_60_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_60_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2038\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1690\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2096\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2142\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1735\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2136\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1701\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1808\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1681\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1927\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1678\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1672\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2086\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1640\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1710\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2195\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1656\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1738\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1743\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1646\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1742\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_67_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_67_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1667\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2034\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2104\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2174\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2005\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2190\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1944\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1973\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1717\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2097\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1761\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1844\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2027\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1653\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2072\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1925\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2044\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_72_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_72_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2214\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1982\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1882\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1707\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1902\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2056\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2150\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2182\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1776\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1795\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2100\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1834\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1780\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2000\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1745\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1807\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2218\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2092\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2204\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2106\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1906\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1774\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1826\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1686\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2132\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1861\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1959\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2117\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1657\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1980\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1775\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1891\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1904\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1746\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1647\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2033\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2110\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1958\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1767\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1943\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2095\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1699\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2157\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2029\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2125\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1870\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1879\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1714\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2222\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1883\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2179\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1648\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2030\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2066\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1889\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2013\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2112\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2197\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1839\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1798\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2009\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1817\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1919\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 25\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1669\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1716\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1711\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2141\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2175\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1884\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2162\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2088\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1713\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1792\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1788\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1786\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_63_piece0 on 10.0.243.141:40637 in memory (size: 26.3 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_63_piece0 on algo-1:36955 in memory (size: 26.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2006\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1803\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2024\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2144\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1966\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_58_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_58_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1637\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1848\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1691\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2061\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1693\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1636\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1675\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1760\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1936\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1813\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1956\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1718\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1845\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1832\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1725\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1794\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1709\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1855\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1999\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2208\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2090\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1703\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2202\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1633\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1946\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2196\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2012\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1805\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1921\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2083\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1799\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1963\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1626\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1740\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1654\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2133\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2206\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1965\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1747\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1887\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1918\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2094\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1790\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1733\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1895\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1631\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1696\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1910\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1779\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2134\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1979\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2017\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2181\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1913\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2021\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1770\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2220\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2151\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1969\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2194\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1726\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1804\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2081\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2102\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1940\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2154\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1704\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2209\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2135\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2028\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2123\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1629\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1829\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2170\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2047\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1932\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1782\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 29\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2037\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1791\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1863\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1651\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1847\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2079\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1684\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1899\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 30\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1892\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1809\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1787\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1712\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1851\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1866\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1901\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1988\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1668\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2041\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1859\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2109\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1814\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1888\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2076\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2120\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2069\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1698\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1783\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2087\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1968\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2184\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1661\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1749\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1939\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1773\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1983\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1873\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2152\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1719\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1955\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2193\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1875\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1635\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1731\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2048\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1929\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2210\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1885\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1688\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1993\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2121\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1990\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2138\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1941\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2016\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1869\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2036\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2019\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1907\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1857\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1721\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1933\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1759\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1800\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1937\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2191\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1952\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2014\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2049\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2180\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1673\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1830\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1821\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_73_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_73_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1950\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1659\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2149\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_74_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_74_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1852\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2163\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1679\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_59_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_59_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1849\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1765\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2082\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1989\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1903\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1692\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2111\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1923\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1663\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2116\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1757\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1945\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1729\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1722\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2101\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1974\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_71_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_71_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1928\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_75_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_75_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2084\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2127\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1894\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2146\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1850\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2018\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1756\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1768\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2065\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1645\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1977\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1868\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2011\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1860\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1676\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2008\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2085\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2131\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2020\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1682\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1947\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1961\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2155\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1748\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2165\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2126\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2113\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 24\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1708\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1991\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2074\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1997\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2186\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2183\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1824\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2143\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1801\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1597\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1835\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2052\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1981\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1810\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2001\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2198\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1931\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_68_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_68_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 74.0 (TID 74) in 115 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 74.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ResultStage 74 (treeReduce at KLLRunner.scala:107) finished in 0.146 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1666\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1987\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1998\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Job 43 finished: treeReduce at KLLRunner.scala:107, took 0.149036 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 22\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1649\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1732\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_64_piece0 on 10.0.243.141:40637 in memory (size: 37.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_64_piece0 on algo-1:36955 in memory (size: 37.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1802\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2201\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2199\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1785\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2042\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2062\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2015\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2219\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1627\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1660\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1685\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2003\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2212\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1871\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1674\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1877\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1728\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2207\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1697\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2105\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1752\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1846\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2130\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1858\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2078\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1670\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1806\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1970\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2089\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1796\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1897\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1753\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1915\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1789\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 26\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2192\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1917\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1644\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1677\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1856\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2007\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2004\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2147\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1841\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2205\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1837\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1694\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2055\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1942\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1683\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1896\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1881\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1912\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1739\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1702\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2139\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2031\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2114\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1911\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1652\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1741\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1736\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2203\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2063\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1750\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1905\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 27\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1890\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2176\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2200\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1643\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2002\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2173\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1893\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1755\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2032\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1957\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2129\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1964\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2185\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1900\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1695\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1642\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1706\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1823\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2091\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1838\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1639\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1985\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1843\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1680\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1665\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1953\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1625\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1763\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2188\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1825\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1992\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1828\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1975\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2160\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1705\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2040\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1827\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1996\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2054\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1771\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1720\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 28\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2171\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2080\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1971\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2057\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1650\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2073\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1730\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2118\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1840\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2167\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2050\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2108\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2122\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1793\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1908\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_69_piece0 on 10.0.243.141:40637 in memory (size: 40.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_69_piece0 on algo-1:36955 in memory (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2071\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2221\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1664\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned shuffle 23\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1797\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1754\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1744\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2010\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2045\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2098\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1777\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2064\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1812\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2022\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1862\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1630\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1751\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1737\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_62_piece0 on algo-1:36955 in memory (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_62_piece0 on 10.0.243.141:40637 in memory (size: 40.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1949\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1984\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2172\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1632\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1769\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2148\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1938\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2168\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2216\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2153\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1867\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1822\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_61_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_61_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1836\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2043\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1831\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2128\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2217\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2211\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2187\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1655\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1994\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1634\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2177\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1930\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2046\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2115\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1926\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2215\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_70_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Removed broadcast_70_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1727\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 2189\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1920\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  ContextCleaner:54 - Cleaned accumulator 1951\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Registering RDD 369 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Got job 44 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Final stage: ResultStage 76 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 75)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 75)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 75 (MapPartitionsRDD[369] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_78 stored as values in memory (estimated size 109.3 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_78_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_78_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 78 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 75 (MapPartitionsRDD[369] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 75.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 75.0 (TID 75, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_78_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 75.0 (TID 75) in 15 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 75.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ShuffleMapStage 75 (collect at AnalysisRunner.scala:313) finished in 0.023 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - waiting: Set(ResultStage 76)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ResultStage 76 (MapPartitionsRDD[372] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_79 stored as values in memory (estimated size 55.0 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_79_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_79_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 79 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 76 (MapPartitionsRDD[372] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 76.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 76.0 (TID 76, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_79_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 31 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 76.0 (TID 76) in 13 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 76.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ResultStage 76 (collect at AnalysisRunner.scala:313) finished in 0.017 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Job 44 finished: collect at AnalysisRunner.scala:313, took 0.043601 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Registering RDD 379 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Got job 45 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Final stage: ResultStage 78 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 77)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 77)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 77 (MapPartitionsRDD[379] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_80 stored as values in memory (estimated size 63.5 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_80_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_80_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 80 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 77 (MapPartitionsRDD[379] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 77.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 77.0 (TID 77, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_80_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 77.0 (TID 77) in 26 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 77.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ShuffleMapStage 77 (countByKey at ColumnProfiler.scala:566) finished in 0.033 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - waiting: Set(ResultStage 78)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ResultStage 78 (ShuffledRDD[380] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_81 stored as values in memory (estimated size 3.2 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_81_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_81_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 81 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 78 (ShuffledRDD[380] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 78.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 78.0 (TID 78, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_81_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 32 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 78.0 (TID 78) in 16 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 78.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ResultStage 78 (countByKey at ColumnProfiler.scala:566) finished in 0.020 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Job 45 finished: countByKey at ColumnProfiler.scala:566, took 0.055850 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Registering RDD 385 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Got job 46 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Final stage: ResultStage 80 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 79)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 79)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 79 (MapPartitionsRDD[385] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_82 stored as values in memory (estimated size 108.7 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_82_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1457.6 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_82_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 82 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 79 (MapPartitionsRDD[385] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 79.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 79.0 (TID 79, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_82_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Finished task 0.0 in stage 79.0 (TID 79) in 81 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Removed TaskSet 79.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - ShuffleMapStage 79 (collect at AnalysisRunner.scala:313) finished in 0.087 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - waiting: Set(ResultStage 80)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting ResultStage 80 (MapPartitionsRDD[388] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_83 stored as values in memory (estimated size 121.1 KB, free 1457.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MemoryStore:54 - Block broadcast_83_piece0 stored as bytes in memory (estimated size 40.7 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_83_piece0 in memory on 10.0.243.141:40637 (size: 40.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  SparkContext:54 - Created broadcast 83 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 80 (MapPartitionsRDD[388] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  YarnScheduler:54 - Adding task set 80.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  TaskSetManager:54 - Starting task 0.0 in stage 80.0 (TID 80, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  BlockManagerInfo:54 - Added broadcast_83_piece0 in memory on algo-1:36955 (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:24 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 33 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 80.0 (TID 80) in 100 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 80.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 80 (collect at AnalysisRunner.scala:313) finished in 0.109 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 46 finished: collect at AnalysisRunner.scala:313, took 0.199563 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 47 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 81 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 81 (MapPartitionsRDD[397] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_84 stored as values in memory (estimated size 69.1 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_84_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1457.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_84_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 84 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 81 (MapPartitionsRDD[397] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 81.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 81.0 (TID 81, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_84_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 81.0 (TID 81) in 46 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 81.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 81 (treeReduce at KLLRunner.scala:107) finished in 0.054 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 47 finished: treeReduce at KLLRunner.scala:107, took 0.055468 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Registering RDD 402 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 48 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 83 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 82)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 82)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 82 (MapPartitionsRDD[402] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_85 stored as values in memory (estimated size 109.3 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_85_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_85_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 85 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 82 (MapPartitionsRDD[402] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 82.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 82.0 (TID 82, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_85_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 82.0 (TID 82) in 16 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 82.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ShuffleMapStage 82 (collect at AnalysisRunner.scala:313) finished in 0.023 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - waiting: Set(ResultStage 83)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 83 (MapPartitionsRDD[405] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_86 stored as values in memory (estimated size 55.0 KB, free 1457.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_86_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_86_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 86 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 83 (MapPartitionsRDD[405] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 83.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 83.0 (TID 83, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_86_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 34 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 83.0 (TID 83) in 11 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 83.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 83 (collect at AnalysisRunner.scala:313) finished in 0.015 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 48 finished: collect at AnalysisRunner.scala:313, took 0.041520 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Registering RDD 412 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 49 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 85 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 84)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 84)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 84 (MapPartitionsRDD[412] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_87 stored as values in memory (estimated size 63.5 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_87_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_87_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 87 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 84 (MapPartitionsRDD[412] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 84.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 84.0 (TID 84, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_87_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 84.0 (TID 84) in 19 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 84.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ShuffleMapStage 84 (countByKey at ColumnProfiler.scala:566) finished in 0.027 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - waiting: Set(ResultStage 85)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 85 (ShuffledRDD[413] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_88 stored as values in memory (estimated size 3.2 KB, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_88_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_88_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 88 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 85 (ShuffledRDD[413] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 85.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 85.0 (TID 85, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_88_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 35 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 85.0 (TID 85) in 12 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 85.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 85 (countByKey at ColumnProfiler.scala:566) finished in 0.016 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 49 finished: countByKey at ColumnProfiler.scala:566, took 0.046332 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Registering RDD 418 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 50 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 87 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 86)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 86)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 86 (MapPartitionsRDD[418] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_89 stored as values in memory (estimated size 108.7 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_89_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_89_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 89 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 86 (MapPartitionsRDD[418] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 86.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 86.0 (TID 86, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_89_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 86.0 (TID 86) in 76 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 86.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ShuffleMapStage 86 (collect at AnalysisRunner.scala:313) finished in 0.082 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - waiting: Set(ResultStage 87)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 87 (MapPartitionsRDD[421] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_90 stored as values in memory (estimated size 121.2 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_90_piece0 stored as bytes in memory (estimated size 40.8 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_90_piece0 in memory on 10.0.243.141:40637 (size: 40.8 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 90 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 87 (MapPartitionsRDD[421] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 87.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 87.0 (TID 87, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_90_piece0 in memory on algo-1:36955 (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 36 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 87.0 (TID 87) in 71 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 87.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 87 (collect at AnalysisRunner.scala:313) finished in 0.080 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 50 finished: collect at AnalysisRunner.scala:313, took 0.166789 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  CodeGenerator:54 - Code generated in 7.700151 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 51 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 88 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 88 (MapPartitionsRDD[430] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_91 stored as values in memory (estimated size 69.1 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_91_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_91_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 91 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 88 (MapPartitionsRDD[430] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 88.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 88.0 (TID 88, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_91_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 88.0 (TID 88) in 57 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 88.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 88 (treeReduce at KLLRunner.scala:107) finished in 0.065 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 51 finished: treeReduce at KLLRunner.scala:107, took 0.066904 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  CodeGenerator:54 - Code generated in 25.806487 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Registering RDD 435 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 52 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 90 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 89)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 89)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 89 (MapPartitionsRDD[435] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_92 stored as values in memory (estimated size 109.3 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_92_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_92_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 92 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 89 (MapPartitionsRDD[435] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 89.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 89.0 (TID 89, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_92_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 89.0 (TID 89) in 69 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 89.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ShuffleMapStage 89 (collect at AnalysisRunner.scala:313) finished in 0.078 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - waiting: Set(ResultStage 90)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ResultStage 90 (MapPartitionsRDD[438] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_93 stored as values in memory (estimated size 55.0 KB, free 1456.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_93_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_93_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Created broadcast 93 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 90 (MapPartitionsRDD[438] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Adding task set 90.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Starting task 0.0 in stage 90.0 (TID 90, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  BlockManagerInfo:54 - Added broadcast_93_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 37 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  TaskSetManager:54 - Finished task 0.0 in stage 90.0 (TID 90) in 18 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  YarnScheduler:54 - Removed TaskSet 90.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - ResultStage 90 (collect at AnalysisRunner.scala:313) finished in 0.026 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Job 52 finished: collect at AnalysisRunner.scala:313, took 0.110615 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Registering RDD 445 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Got job 53 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Final stage: ResultStage 92 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 91)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 91)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 91 (MapPartitionsRDD[445] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:25 INFO  MemoryStore:54 - Block broadcast_94 stored as values in memory (estimated size 63.5 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_94_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1456.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_94_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 94 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 91 (MapPartitionsRDD[445] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 91.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 91.0 (TID 91, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_94_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 91.0 (TID 91) in 26 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 91.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ShuffleMapStage 91 (countByKey at ColumnProfiler.scala:566) finished in 0.039 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - waiting: Set(ResultStage 92)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 92 (ShuffledRDD[446] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_95 stored as values in memory (estimated size 3.2 KB, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_95_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1456.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_95_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 95 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 92 (ShuffledRDD[446] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 92.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 92.0 (TID 92, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_95_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 38 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 92.0 (TID 92) in 31 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 92.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 92 (countByKey at ColumnProfiler.scala:566) finished in 0.036 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 53 finished: countByKey at ColumnProfiler.scala:566, took 0.077645 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Registering RDD 451 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Got job 54 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Final stage: ResultStage 94 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 93)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 93)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 93 (MapPartitionsRDD[451] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_96 stored as values in memory (estimated size 108.7 KB, free 1456.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_96_piece0 stored as bytes in memory (estimated size 37.9 KB, free 1456.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_96_piece0 in memory on 10.0.243.141:40637 (size: 37.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 96 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 93 (MapPartitionsRDD[451] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 93.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 93.0 (TID 93, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_96_piece0 in memory on algo-1:36955 (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 93.0 (TID 93) in 114 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 93.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ShuffleMapStage 93 (collect at AnalysisRunner.scala:313) finished in 0.121 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - waiting: Set(ResultStage 94)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 94 (MapPartitionsRDD[454] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_97 stored as values in memory (estimated size 121.4 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_97_piece0 stored as bytes in memory (estimated size 40.8 KB, free 1456.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_97_piece0 in memory on 10.0.243.141:40637 (size: 40.8 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 97 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 94 (MapPartitionsRDD[454] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 94.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 94.0 (TID 94, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_97_piece0 in memory on algo-1:36955 (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 39 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 94.0 (TID 94) in 66 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 94.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 94 (collect at AnalysisRunner.scala:313) finished in 0.073 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 54 finished: collect at AnalysisRunner.scala:313, took 0.199128 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Starting job: treeReduce at KLLRunner.scala:107\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Got job 55 (treeReduce at KLLRunner.scala:107) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Final stage: ResultStage 95 (treeReduce at KLLRunner.scala:107)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Parents of final stage: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Missing parents: List()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 95 (MapPartitionsRDD[463] at treeReduce at KLLRunner.scala:107), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_98 stored as values in memory (estimated size 69.1 KB, free 1456.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_98_piece0 stored as bytes in memory (estimated size 26.4 KB, free 1456.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_98_piece0 in memory on 10.0.243.141:40637 (size: 26.4 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 98 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 95 (MapPartitionsRDD[463] at treeReduce at KLLRunner.scala:107) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 95.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 95.0 (TID 95, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8363 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_98_piece0 in memory on algo-1:36955 (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 95.0 (TID 95) in 46 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 95.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 95 (treeReduce at KLLRunner.scala:107) finished in 0.053 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 55 finished: treeReduce at KLLRunner.scala:107, took 0.054970 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2811\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2546\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2585\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2690\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2822\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2776\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2677\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_98_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_98_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2636\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2692\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2759\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2805\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2490\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2373\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2582\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2642\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2240\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2619\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2718\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_77_piece0 on 10.0.243.141:40637 in memory (size: 26.3 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_77_piece0 on algo-1:36955 in memory (size: 26.3 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2376\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2405\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2534\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2666\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2340\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2802\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2648\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2299\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2542\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2523\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2866\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2650\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2639\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_79_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Starting job: collect at AnalysisRunner.scala:313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_79_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Registering RDD 468 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Got job 56 (collect at AnalysisRunner.scala:313) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Final stage: ResultStage 97 (collect at AnalysisRunner.scala:313)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 96)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 96)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 96 (MapPartitionsRDD[468] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2637\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2777\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2464\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2456\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2460\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2663\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2838\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2553\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2686\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2828\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2301\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2632\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2285\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2474\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2513\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2595\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2365\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2430\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2810\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2779\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2302\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2672\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2561\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2325\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2714\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2833\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2863\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2372\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2614\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2781\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2537\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2393\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2540\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2451\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2539\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2798\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2772\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2799\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2578\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2547\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2476\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2867\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2654\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2277\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_99 stored as values in memory (estimated size 109.3 KB, free 1456.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_80_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_80_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_99_piece0 stored as bytes in memory (estimated size 37.8 KB, free 1456.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2346\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_99_piece0 in memory on 10.0.243.141:40637 (size: 37.8 KB, free: 1458.0 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_96_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_96_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_78_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 99 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 96 (MapPartitionsRDD[468] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 96.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_78_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 96.0 (TID 96, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2767\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2706\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2387\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2273\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2836\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2326\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2472\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2600\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2829\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2274\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2577\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2784\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2342\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2711\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2489\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2525\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2492\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2508\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2253\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2532\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2433\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2671\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2755\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2800\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2549\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2422\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2688\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2797\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2504\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2826\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2318\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2515\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2244\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2406\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2256\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2860\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2804\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2611\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 35\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2452\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2749\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2384\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2450\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2813\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2383\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2300\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2780\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2436\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2748\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2374\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2599\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2738\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2229\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2678\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2429\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2726\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2322\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2251\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2359\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2501\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2526\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2321\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2370\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2694\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2224\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2239\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2700\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 34\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2728\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2305\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2703\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2591\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2739\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2470\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2506\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2788\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2459\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2763\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2511\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2869\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2227\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2316\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2327\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2463\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2428\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2709\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2418\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2427\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2505\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2259\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2343\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_95_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.1 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_99_piece0 in memory on algo-1:36955 (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_95_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2831\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2750\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2308\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2230\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2481\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2279\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2565\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2708\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2312\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2246\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2878\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2320\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2786\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2727\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2440\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2486\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2669\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2271\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2819\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2529\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2832\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2241\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2243\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2500\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2423\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2361\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2286\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2746\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2627\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2385\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2626\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2261\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2352\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2628\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2865\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2499\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2717\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2263\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2495\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2293\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2791\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2358\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2351\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2790\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2403\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 32\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2332\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2514\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2733\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2441\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2715\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2757\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2425\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2268\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2680\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2536\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2768\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2729\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2814\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2705\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2778\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2858\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2357\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2741\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2720\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2520\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2521\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2652\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2421\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2735\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2769\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2588\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2558\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2656\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2266\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2247\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2453\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2551\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2487\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2594\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2290\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2404\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2432\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2330\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2775\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2304\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2531\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2584\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2854\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2264\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2349\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2712\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2475\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2670\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2817\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2856\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2573\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2395\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2282\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2771\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2625\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_85_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_85_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2339\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2386\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2621\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2297\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2319\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2355\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2563\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2334\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2462\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2789\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2658\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2538\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2289\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2550\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2687\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2761\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2783\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2522\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2502\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2616\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2455\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2389\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2651\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 37\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2434\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2519\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2721\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2606\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2556\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 38\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2458\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2392\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2407\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2646\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2807\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2317\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2341\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2415\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2262\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2257\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2509\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2624\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_94_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_94_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2846\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2337\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2445\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2226\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2278\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2424\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2485\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2675\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2853\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2568\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2612\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2444\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2232\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2408\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2587\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2766\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_91_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_91_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2696\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2635\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2723\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2830\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2841\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2615\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2604\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 39\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2562\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 31\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2442\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2533\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2701\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2367\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2809\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2528\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2825\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2348\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2724\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2702\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2313\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2461\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2610\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2640\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2795\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2443\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2518\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2734\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2876\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2249\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2527\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2576\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2402\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2559\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2793\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2695\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2602\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 96.0 (TID 96) in 41 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 96.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_82_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ShuffleMapStage 96 (collect at AnalysisRunner.scala:313) finished in 0.050 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - waiting: Set(ResultStage 97)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 97 (MapPartitionsRDD[471] at collect at AnalysisRunner.scala:313), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_82_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_100 stored as values in memory (estimated size 55.0 KB, free 1456.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_100_piece0 stored as bytes in memory (estimated size 16.9 KB, free 1456.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2483\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_100_piece0 in memory on 10.0.243.141:40637 (size: 16.9 KB, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2471\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2605\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2400\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2311\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2722\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2719\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2740\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2225\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2623\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2862\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2541\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2283\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2693\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2685\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2242\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2668\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 100 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2837\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 97 (MapPartitionsRDD[471] at collect at AnalysisRunner.scala:313) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 97.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 97.0 (TID 97, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_88_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_88_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2667\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2382\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2859\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2684\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2574\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2644\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_81_piece0 on 10.0.243.141:40637 in memory (size: 1924.0 B, free: 1458.2 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_81_piece0 on algo-1:36955 in memory (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2835\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2447\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2491\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2388\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2397\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2596\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2223\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2478\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2742\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2281\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2378\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2634\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2572\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2683\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2731\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2681\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2399\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2691\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2324\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2655\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_92_piece0 on 10.0.243.141:40637 in memory (size: 37.8 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_100_piece0 in memory on algo-1:36955 (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_92_piece0 on algo-1:36955 in memory (size: 37.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2710\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2288\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_97_piece0 on 10.0.243.141:40637 in memory (size: 40.8 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 40 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_97_piece0 on algo-1:36955 in memory (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2629\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2785\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2377\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2653\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2250\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2689\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2493\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2857\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2554\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2510\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_89_piece0 on 10.0.243.141:40637 in memory (size: 37.9 KB, free: 1458.3 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 97.0 (TID 97) in 27 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 97.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 97 (collect at AnalysisRunner.scala:313) finished in 0.033 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 56 finished: collect at AnalysisRunner.scala:313, took 0.085782 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_89_piece0 on algo-1:36955 in memory (size: 37.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2564\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2659\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2674\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2586\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2848\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2260\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2245\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2545\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2834\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2234\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2516\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2704\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_93_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_93_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2665\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2661\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2592\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2437\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2618\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2449\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2335\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2488\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2394\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2336\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2579\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2410\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2664\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2645\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2815\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2824\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2296\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2840\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2417\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2512\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2238\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2248\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2331\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2345\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2737\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2816\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2431\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2466\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2617\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2682\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2580\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2753\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2276\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2454\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2231\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2530\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2609\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2333\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2607\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2350\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2743\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2435\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2328\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2570\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2871\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2369\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2751\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2419\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2314\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2233\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2747\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2713\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2390\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2874\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2792\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2875\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2760\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2861\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2448\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2839\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2439\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2469\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2303\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2543\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2295\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2391\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2398\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2603\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2375\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2630\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2414\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2473\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2581\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2457\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2482\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2638\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2467\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2864\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2548\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2284\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2315\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2803\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2344\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2657\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2870\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2673\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_86_piece0 on 10.0.243.141:40637 in memory (size: 16.9 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_86_piece0 on algo-1:36955 in memory (size: 16.9 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2601\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2855\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2347\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2503\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2732\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2362\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2818\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2237\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2265\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2764\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2873\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2847\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2291\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2589\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2679\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2235\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2236\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2765\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2643\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2827\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2844\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2381\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2730\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2699\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2535\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2660\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2292\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2823\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2773\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2845\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2356\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2363\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2770\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2287\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2465\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2507\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2228\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2409\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2613\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2484\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2787\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2323\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2796\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2294\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2309\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2571\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2401\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2446\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2631\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2754\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2842\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2676\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2364\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2552\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2480\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2752\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 33\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2498\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2597\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2338\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2774\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2258\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2396\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2593\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2477\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2725\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2821\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2420\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2598\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2868\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2479\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2716\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2794\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2298\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2782\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2275\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2567\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2354\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2412\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2438\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2416\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2494\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_87_piece0 on 10.0.243.141:40637 in memory (size: 24.7 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_87_piece0 on algo-1:36955 in memory (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_90_piece0 on 10.0.243.141:40637 in memory (size: 40.8 KB, free: 1458.4 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_90_piece0 on algo-1:36955 in memory (size: 40.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2697\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2497\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2255\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2622\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2820\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2254\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2306\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2544\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2272\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2269\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2647\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2555\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_83_piece0 on 10.0.243.141:40637 in memory (size: 40.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_83_piece0 on algo-1:36955 in memory (size: 40.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2756\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2411\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2608\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2280\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2366\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2758\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2267\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2310\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2371\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2426\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2849\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2620\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2801\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2557\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2736\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2496\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2307\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2707\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2808\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2379\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2360\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2877\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2368\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2413\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2633\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2380\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2762\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2662\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2468\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2843\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2812\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2252\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2744\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2569\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2590\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Starting job: countByKey at ColumnProfiler.scala:566\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned shuffle 36\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2270\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2641\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2745\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2560\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2806\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2517\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2524\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2698\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2872\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2583\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2575\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Registering RDD 478 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Got job 57 (countByKey at ColumnProfiler.scala:566) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_84_piece0 on 10.0.243.141:40637 in memory (size: 26.4 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Final stage: ResultStage 99 (countByKey at ColumnProfiler.scala:566)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 98)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 98)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 98 (MapPartitionsRDD[478] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Removed broadcast_84_piece0 on algo-1:36955 in memory (size: 26.4 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_101 stored as values in memory (estimated size 63.5 KB, free 1457.9 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2353\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2329\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ContextCleaner:54 - Cleaned accumulator 2649\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_101_piece0 stored as bytes in memory (estimated size 24.7 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_101_piece0 in memory on 10.0.243.141:40637 (size: 24.7 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 101 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 98 (MapPartitionsRDD[478] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 98.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 98.0 (TID 98, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_101_piece0 in memory on algo-1:36955 (size: 24.7 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 98.0 (TID 98) in 19 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 98.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ShuffleMapStage 98 (countByKey at ColumnProfiler.scala:566) finished in 0.025 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - waiting: Set(ResultStage 99)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 99 (ShuffledRDD[479] at countByKey at ColumnProfiler.scala:566), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_102 stored as values in memory (estimated size 3.2 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_102_piece0 stored as bytes in memory (estimated size 1924.0 B, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_102_piece0 in memory on 10.0.243.141:40637 (size: 1924.0 B, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 102 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 99 (ShuffledRDD[479] at countByKey at ColumnProfiler.scala:566) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 99.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 99.0 (TID 99, algo-1, executor 1, partition 0, NODE_LOCAL, 7660 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_102_piece0 in memory on algo-1:36955 (size: 1924.0 B, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 41 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 99.0 (TID 99) in 14 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 99.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 99 (countByKey at ColumnProfiler.scala:566) finished in 0.019 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 57 finished: countByKey at ColumnProfiler.scala:566, took 0.047817 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ConstraintGenerator:45 - Generating Constraints:\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  ConstraintGenerator:50 - Constraints: {\n",
      "  \"version\" : 0.0,\n",
      "  \"features\" : [ {\n",
      "    \"name\" : \"Churn\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Account Length\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"VMail Message\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Day Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Day Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Eve Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Eve Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Night Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Night Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Intl Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Intl Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"CustServ Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AK\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AR\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AZ\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CO\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CT\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_DC\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_DE\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_FL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_GA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_HI\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_ID\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IN\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_KS\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_KY\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_LA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MD\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_ME\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MI\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MN\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MO\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MS\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MT\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NC\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_ND\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NE\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NH\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NJ\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NM\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NV\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_NY\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_OH\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_OK\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_OR\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_PA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_RI\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_SC\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_SD\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_TN\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_TX\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_UT\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_VA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_VT\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_WA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_WI\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_WV\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_WY\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Area Code_408\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Area Code_415\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Area Code_510\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Int'l Plan_no\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Int'l Plan_yes\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"VMail Plan_no\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"VMail Plan_yes\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"completeness\" : 1.0,\n",
      "    \"num_constraints\" : {\n",
      "      \"is_non_negative\" : true\n",
      "    }\n",
      "  } ],\n",
      "  \"monitoring_config\" : {\n",
      "    \"evaluate_constraints\" : \"Enabled\",\n",
      "    \"emit_metrics\" : \"Enabled\",\n",
      "    \"datatype_check_threshold\" : 1.0,\n",
      "    \"domain_content_threshold\" : 1.0,\n",
      "    \"distribution_constraints\" : {\n",
      "      \"perform_comparison\" : \"Enabled\",\n",
      "      \"comparison_threshold\" : 0.1,\n",
      "      \"comparison_method\" : \"Robust\"\n",
      "    }\n",
      "  }\u001b[0m\n",
      "\u001b[34m}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  FileUtil:29 - Write to file constraints.json at path /opt/ml/processing/output.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  StatsGenerator:65 - Generating Stats:\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  CodeGenerator:54 - Code generated in 6.104586 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  CodeGenerator:54 - Code generated in 5.364187 ms\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Starting job: count at StatsGenerator.scala:67\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Registering RDD 484 (count at StatsGenerator.scala:67)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Got job 58 (count at StatsGenerator.scala:67) with 1 output partitions\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Final stage: ResultStage 101 (count at StatsGenerator.scala:67)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 100)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 100)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 100 (MapPartitionsRDD[484] at count at StatsGenerator.scala:67), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_103 stored as values in memory (estimated size 64.2 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_103_piece0 stored as bytes in memory (estimated size 24.8 KB, free 1457.8 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_103_piece0 in memory on 10.0.243.141:40637 (size: 24.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 103 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ShuffleMapStage 100 (MapPartitionsRDD[484] at count at StatsGenerator.scala:67) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 100.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 100.0 (TID 100, algo-1, executor 1, partition 0, PROCESS_LOCAL, 8352 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_103_piece0 in memory on algo-1:36955 (size: 24.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 100.0 (TID 100) in 32 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 100.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ShuffleMapStage 100 (count at StatsGenerator.scala:67) finished in 0.038 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - looking for newly runnable stages\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - running: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - waiting: Set(ResultStage 101)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - failed: Set()\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting ResultStage 101 (MapPartitionsRDD[487] at count at StatsGenerator.scala:67), which has no missing parents\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_104 stored as values in memory (estimated size 7.4 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MemoryStore:54 - Block broadcast_104_piece0 stored as bytes in memory (estimated size 3.8 KB, free 1457.7 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_104_piece0 in memory on 10.0.243.141:40637 (size: 3.8 KB, free: 1458.5 MB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  SparkContext:54 - Created broadcast 104 from broadcast at DAGScheduler.scala:1039\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 101 (MapPartitionsRDD[487] at count at StatsGenerator.scala:67) (first 15 tasks are for partitions Vector(0))\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Adding task set 101.0 with 1 tasks\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Starting task 0.0 in stage 101.0 (TID 101, algo-1, executor 1, partition 0, NODE_LOCAL, 7765 bytes)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  BlockManagerInfo:54 - Added broadcast_104_piece0 in memory on algo-1:36955 (size: 3.8 KB, free: 5.8 GB)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 42 to 10.0.243.141:45068\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  TaskSetManager:54 - Finished task 0.0 in stage 101.0 (TID 101) in 22 ms on algo-1 (executor 1) (1/1)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  YarnScheduler:54 - Removed TaskSet 101.0, whose tasks have all completed, from pool \u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - ResultStage 101 (count at StatsGenerator.scala:67) finished in 0.038 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:26 INFO  DAGScheduler:54 - Job 58 finished: count at StatsGenerator.scala:67, took 0.080019 s\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  StatsGenerator:70 - Stats: {\n",
      "  \"version\" : 0.0,\n",
      "  \"dataset\" : {\n",
      "    \"item_count\" : 2333\n",
      "  },\n",
      "  \"features\" : [ {\n",
      "    \"name\" : \"Churn\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.1393056150878697,\n",
      "      \"sum\" : 325.0,\n",
      "      \"std_dev\" : 0.34626515951342846,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2008.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 325.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Account Length\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 101.2768966995285,\n",
      "      \"sum\" : 236279.0,\n",
      "      \"std_dev\" : 39.552442167470566,\n",
      "      \"min\" : 1.0,\n",
      "      \"max\" : 243.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 1.0,\n",
      "            \"upper_bound\" : 25.2,\n",
      "            \"count\" : 70.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 25.2,\n",
      "            \"upper_bound\" : 49.4,\n",
      "            \"count\" : 150.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 49.4,\n",
      "            \"upper_bound\" : 73.6,\n",
      "            \"count\" : 353.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 73.6,\n",
      "            \"upper_bound\" : 97.8,\n",
      "            \"count\" : 518.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 97.8,\n",
      "            \"upper_bound\" : 122.0,\n",
      "            \"count\" : 538.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 122.0,\n",
      "            \"upper_bound\" : 146.2,\n",
      "            \"count\" : 401.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 146.2,\n",
      "            \"upper_bound\" : 170.4,\n",
      "            \"count\" : 208.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 170.4,\n",
      "            \"upper_bound\" : 194.6,\n",
      "            \"count\" : 72.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 194.6,\n",
      "            \"upper_bound\" : 218.8,\n",
      "            \"count\" : 19.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 218.8,\n",
      "            \"upper_bound\" : 243.0,\n",
      "            \"count\" : 4.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 119.0, 100.0, 111.0, 181.0, 95.0, 104.0, 70.0, 120.0, 88.0, 111.0, 33.0, 106.0, 54.0, 87.0, 94.0, 135.0, 107.0, 159.0, 106.0, 136.0, 116.0, 115.0, 103.0, 95.0, 115.0, 143.0, 48.0, 94.0, 153.0, 94.0, 107.0, 91.0, 141.0, 58.0, 49.0, 41.0, 137.0, 111.0, 71.0, 43.0, 97.0, 3.0, 124.0, 86.0, 87.0, 83.0, 67.0, 46.0, 129.0, 90.0, 97.0, 87.0, 141.0, 136.0, 88.0, 170.0, 44.0, 121.0, 111.0, 105.0, 112.0, 73.0, 147.0, 66.0, 136.0, 119.0, 135.0, 102.0, 169.0, 60.0, 73.0, 83.0, 90.0, 148.0, 59.0, 152.0, 136.0, 112.0, 122.0, 44.0, 122.0, 89.0, 176.0, 64.0, 112.0, 133.0, 52.0, 91.0, 127.0, 153.0, 117.0, 163.0, 76.0, 80.0, 136.0, 91.0, 143.0, 125.0, 126.0, 87.0, 119.0, 13.0, 138.0, 159.0, 111.0, 46.0, 68.0, 107.0, 70.0, 215.0, 22.0, 122.0, 73.0, 75.0, 87.0, 148.0, 105.0, 182.0, 139.0, 105.0, 166.0, 60.0, 76.0, 28.0, 94.0, 146.0, 101.0, 132.0, 93.0, 105.0, 100.0, 134.0, 63.0, 126.0, 166.0, 160.0, 162.0, 70.0, 116.0, 75.0, 74.0, 115.0, 42.0, 132.0, 171.0, 135.0, 99.0, 27.0, 139.0, 76.0, 123.0, 54.0, 70.0, 163.0, 96.0, 62.0, 115.0, 97.0, 137.0, 82.0, 118.0, 64.0, 186.0, 117.0, 117.0, 116.0, 164.0, 103.0, 137.0, 97.0, 144.0, 96.0, 183.0, 42.0, 100.0, 131.0, 88.0, 91.0, 104.0, 63.0, 159.0, 147.0, 123.0, 100.0, 105.0, 163.0, 90.0, 125.0, 64.0, 113.0, 101.0, 123.0, 212.0, 73.0, 44.0, 96.0, 74.0, 77.0, 120.0, 122.0, 87.0, 52.0, 48.0, 61.0, 141.0, 170.0, 17.0, 162.0, 85.0, 160.0, 29.0, 91.0, 96.0, 104.0, 95.0, 84.0, 157.0, 165.0, 57.0, 95.0, 51.0, 97.0, 13.0, 50.0, 46.0, 121.0, 68.0, 72.0, 82.0, 38.0, 41.0, 96.0, 129.0, 31.0, 122.0, 51.0, 109.0, 161.0, 72.0, 65.0, 129.0, 137.0, 48.0, 134.0, 125.0, 153.0, 103.0, 45.0, 80.0, 57.0, 94.0, 59.0, 72.0, 62.0, 155.0, 96.0, 77.0, 58.0, 134.0, 24.0, 158.0, 89.0, 138.0, 61.0, 123.0, 87.0, 74.0, 37.0, 105.0, 56.0, 64.0, 202.0, 91.0, 120.0, 89.0, 95.0, 92.0, 45.0, 106.0, 125.0, 129.0, 159.0, 99.0 ], [ 1.0, 1.0, 1.0, 1.0, 3.0, 3.0, 5.0, 6.0, 7.0, 9.0, 9.0, 10.0, 11.0, 13.0, 13.0, 13.0, 15.0, 16.0, 16.0, 16.0, 17.0, 19.0, 19.0, 20.0, 21.0, 21.0, 21.0, 22.0, 23.0, 24.0, 24.0, 25.0, 26.0, 27.0, 27.0, 28.0, 28.0, 29.0, 30.0, 31.0, 31.0, 32.0, 32.0, 32.0, 33.0, 33.0, 34.0, 35.0, 35.0, 35.0, 36.0, 36.0, 36.0, 36.0, 37.0, 37.0, 37.0, 38.0, 38.0, 39.0, 39.0, 39.0, 40.0, 40.0, 40.0, 40.0, 41.0, 41.0, 41.0, 41.0, 42.0, 42.0, 43.0, 43.0, 43.0, 44.0, 44.0, 44.0, 45.0, 45.0, 45.0, 45.0, 46.0, 46.0, 46.0, 46.0, 47.0, 47.0, 47.0, 48.0, 48.0, 48.0, 48.0, 49.0, 49.0, 50.0, 50.0, 51.0, 51.0, 51.0, 51.0, 52.0, 52.0, 52.0, 52.0, 52.0, 53.0, 53.0, 53.0, 53.0, 54.0, 54.0, 54.0, 54.0, 54.0, 55.0, 55.0, 55.0, 55.0, 55.0, 55.0, 55.0, 56.0, 56.0, 56.0, 56.0, 57.0, 57.0, 57.0, 57.0, 57.0, 57.0, 58.0, 58.0, 58.0, 58.0, 58.0, 59.0, 59.0, 59.0, 59.0, 59.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 60.0, 61.0, 61.0, 61.0, 61.0, 61.0, 61.0, 62.0, 62.0, 62.0, 62.0, 62.0, 62.0, 62.0, 63.0, 63.0, 63.0, 63.0, 63.0, 63.0, 63.0, 63.0, 64.0, 64.0, 64.0, 64.0, 64.0, 64.0, 64.0, 64.0, 65.0, 65.0, 65.0, 65.0, 65.0, 65.0, 65.0, 65.0, 66.0, 66.0, 66.0, 66.0, 66.0, 66.0, 66.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 69.0, 69.0, 69.0, 69.0, 69.0, 69.0, 70.0, 70.0, 70.0, 70.0, 70.0, 70.0, 70.0, 70.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 72.0, 72.0, 72.0, 72.0, 72.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 77.0, 77.0, 77.0, 77.0, 77.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 131.0, 131.0, 131.0, 131.0, 131.0, 132.0, 132.0, 132.0, 132.0, 132.0, 132.0, 132.0, 132.0, 132.0, 133.0, 133.0, 133.0, 133.0, 133.0, 133.0, 134.0, 134.0, 134.0, 134.0, 135.0, 135.0, 135.0, 135.0, 136.0, 136.0, 136.0, 136.0, 136.0, 136.0, 136.0, 137.0, 137.0, 137.0, 137.0, 137.0, 137.0, 137.0, 138.0, 138.0, 138.0, 138.0, 138.0, 138.0, 138.0, 139.0, 139.0, 139.0, 139.0, 139.0, 139.0, 139.0, 140.0, 140.0, 140.0, 140.0, 140.0, 141.0, 141.0, 141.0, 141.0, 141.0, 141.0, 141.0, 141.0, 142.0, 142.0, 142.0, 142.0, 142.0, 143.0, 143.0, 143.0, 143.0, 143.0, 144.0, 144.0, 144.0, 144.0, 144.0, 144.0, 145.0, 145.0, 145.0, 145.0, 145.0, 145.0, 146.0, 146.0, 146.0, 146.0, 146.0, 146.0, 147.0, 147.0, 147.0, 147.0, 147.0, 148.0, 148.0, 148.0, 148.0, 148.0, 148.0, 148.0, 148.0, 148.0, 148.0, 149.0, 149.0, 149.0, 149.0, 149.0, 149.0, 149.0, 150.0, 150.0, 150.0, 150.0, 151.0, 151.0, 151.0, 151.0, 151.0, 152.0, 152.0, 153.0, 153.0, 153.0, 154.0, 154.0, 154.0, 154.0, 155.0, 155.0, 155.0, 155.0, 155.0, 155.0, 156.0, 156.0, 156.0, 157.0, 157.0, 157.0, 157.0, 158.0, 158.0, 158.0, 159.0, 159.0, 159.0, 160.0, 160.0, 160.0, 160.0, 161.0, 161.0, 161.0, 161.0, 161.0, 162.0, 162.0, 163.0, 163.0, 163.0, 164.0, 164.0, 165.0, 165.0, 165.0, 166.0, 166.0, 166.0, 166.0, 167.0, 168.0, 168.0, 169.0, 169.0, 170.0, 170.0, 172.0, 172.0, 172.0, 173.0, 173.0, 174.0, 174.0, 174.0, 176.0, 176.0, 177.0, 177.0, 177.0, 179.0, 179.0, 180.0, 180.0, 181.0, 181.0, 182.0, 183.0, 183.0, 184.0, 184.0, 185.0, 185.0, 189.0, 189.0, 190.0, 190.0, 192.0, 193.0, 193.0, 195.0, 197.0, 201.0, 204.0, 205.0, 209.0, 210.0, 217.0, 224.0, 225.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"VMail Message\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 8.214316330904415,\n",
      "      \"sum\" : 19164.0,\n",
      "      \"std_dev\" : 13.776907846587017,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 51.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 5.1,\n",
      "            \"count\" : 1684.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 5.1,\n",
      "            \"upper_bound\" : 10.2,\n",
      "            \"count\" : 2.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 10.2,\n",
      "            \"upper_bound\" : 15.3,\n",
      "            \"count\" : 15.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 15.3,\n",
      "            \"upper_bound\" : 20.4,\n",
      "            \"count\" : 52.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 20.4,\n",
      "            \"upper_bound\" : 25.5,\n",
      "            \"count\" : 127.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 25.5,\n",
      "            \"upper_bound\" : 30.6,\n",
      "            \"count\" : 171.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 30.6,\n",
      "            \"upper_bound\" : 35.7,\n",
      "            \"count\" : 135.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 35.7,\n",
      "            \"upper_bound\" : 40.8,\n",
      "            \"count\" : 106.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 40.8,\n",
      "            \"upper_bound\" : 45.9,\n",
      "            \"count\" : 32.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 45.9,\n",
      "            \"upper_bound\" : 51.0,\n",
      "            \"count\" : 9.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 19.0, 0.0, 0.0, 40.0, 36.0, 0.0, 0.0, 24.0, 0.0, 0.0, 35.0, 0.0, 0.0, 0.0, 0.0, 41.0, 0.0, 0.0, 0.0, 24.0, 0.0, 33.0, 0.0, 37.0, 0.0, 0.0, 43.0, 0.0, 31.0, 28.0, 0.0, 37.0, 0.0, 0.0, 28.0, 34.0, 0.0, 0.0, 0.0, 35.0, 0.0, 36.0, 0.0, 29.0, 0.0, 30.0, 35.0, 0.0, 33.0, 0.0, 0.0, 0.0, 37.0, 0.0, 0.0, 0.0, 0.0, 24.0, 0.0, 24.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 26.0, 0.0, 0.0, 29.0, 20.0, 33.0, 0.0, 0.0, 0.0, 23.0, 0.0, 0.0, 22.0, 0.0, 32.0, 0.0, 34.0, 0.0, 22.0, 0.0, 0.0, 0.0, 0.0, 27.0, 0.0, 0.0, 29.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 23.0, 0.0, 28.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 19.0, 16.0, 0.0, 0.0, 0.0, 38.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 19.0, 46.0, 0.0, 16.0, 0.0, 0.0, 0.0, 22.0, 0.0, 0.0, 0.0, 0.0, 0.0, 39.0, 0.0, 25.0, 33.0, 0.0, 0.0, 0.0, 19.0, 0.0, 36.0, 0.0, 26.0, 0.0, 0.0, 35.0, 30.0, 0.0, 50.0, 28.0, 0.0, 26.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 49.0, 15.0, 0.0, 0.0, 27.0, 0.0, 23.0, 0.0, 0.0, 0.0, 0.0, 21.0, 0.0, 0.0, 0.0, 0.0, 26.0, 33.0, 0.0, 0.0, 0.0, 0.0, 24.0, 36.0, 40.0, 0.0, 0.0, 35.0, 0.0, 0.0, 0.0, 37.0, 0.0, 0.0, 0.0, 27.0, 0.0, 0.0, 0.0, 30.0, 0.0, 28.0, 0.0, 0.0, 0.0, 0.0, 35.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 28.0, 0.0, 0.0, 0.0, 38.0, 0.0, 29.0, 31.0, 0.0, 34.0, 0.0, 0.0, 0.0, 24.0, 0.0, 38.0, 0.0, 0.0, 0.0, 0.0, 0.0, 30.0, 21.0, 0.0, 30.0, 32.0, 0.0, 0.0, 0.0, 17.0, 27.0, 0.0, 0.0, 0.0, 0.0, 31.0, 0.0, 31.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 33.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9.0, 12.0, 13.0, 14.0, 14.0, 14.0, 15.0, 15.0, 16.0, 16.0, 16.0, 16.0, 17.0, 17.0, 17.0, 18.0, 18.0, 18.0, 19.0, 19.0, 19.0, 19.0, 19.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 21.0, 21.0, 21.0, 21.0, 21.0, 21.0, 21.0, 21.0, 21.0, 22.0, 22.0, 22.0, 22.0, 22.0, 22.0, 22.0, 22.0, 22.0, 22.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 23.0, 24.0, 24.0, 24.0, 24.0, 24.0, 24.0, 24.0, 24.0, 24.0, 24.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 27.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 28.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 29.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 30.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 31.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 32.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 33.0, 34.0, 34.0, 34.0, 34.0, 34.0, 34.0, 34.0, 34.0, 34.0, 35.0, 35.0, 35.0, 35.0, 35.0, 35.0, 35.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 36.0, 37.0, 37.0, 37.0, 37.0, 37.0, 37.0, 37.0, 37.0, 37.0, 38.0, 38.0, 38.0, 38.0, 38.0, 38.0, 38.0, 38.0, 38.0, 39.0, 39.0, 39.0, 39.0, 39.0, 39.0, 39.0, 39.0, 39.0, 39.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 41.0, 41.0, 41.0, 42.0, 42.0, 42.0, 42.0, 42.0, 43.0, 43.0, 43.0, 44.0, 44.0, 45.0, 45.0, 46.0, 47.0, 50.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Day Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 180.22648949849963,\n",
      "      \"sum\" : 420468.3999999996,\n",
      "      \"std_dev\" : 53.987178959901556,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 350.8,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 35.08,\n",
      "            \"count\" : 14.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 35.08,\n",
      "            \"upper_bound\" : 70.16,\n",
      "            \"count\" : 48.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 70.16,\n",
      "            \"upper_bound\" : 105.24000000000001,\n",
      "            \"count\" : 130.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 105.24000000000001,\n",
      "            \"upper_bound\" : 140.32,\n",
      "            \"count\" : 318.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 140.32,\n",
      "            \"upper_bound\" : 175.4,\n",
      "            \"count\" : 565.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 175.4,\n",
      "            \"upper_bound\" : 210.48000000000002,\n",
      "            \"count\" : 587.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 210.48000000000002,\n",
      "            \"upper_bound\" : 245.56,\n",
      "            \"count\" : 423.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 245.56,\n",
      "            \"upper_bound\" : 280.64,\n",
      "            \"count\" : 180.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 280.64,\n",
      "            \"upper_bound\" : 315.72,\n",
      "            \"count\" : 58.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 315.72,\n",
      "            \"upper_bound\" : 350.8,\n",
      "            \"count\" : 10.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 178.1, 160.3, 197.1, 105.2, 283.1, 113.6, 232.1, 212.7, 73.3, 176.9, 161.9, 128.6, 190.5, 223.2, 157.9, 173.1, 273.5, 275.8, 119.2, 174.6, 133.3, 145.0, 150.6, 220.2, 109.7, 155.4, 172.0, 235.6, 218.5, 92.7, 90.7, 162.3, 146.5, 210.1, 214.4, 194.4, 237.3, 255.9, 197.9, 200.2, 120.8, 118.1, 131.8, 225.4, 205.2, 272.5, 181.1, 122.2, 119.6, 109.6, 112.7, 136.3, 185.4, 199.6, 218.2, 259.9, 143.2, 218.2, 249.8, 274.7, 167.8, 182.3, 157.0, 207.7, 250.2, 81.9, 246.8, 103.1, 147.2, 252.7, 192.2, 226.4, 145.5, 178.3, 133.1, 214.6, 203.9, 185.4, 140.0, 240.3, 134.2, 141.1, 47.4, 200.4, 167.6, 221.1, 165.5, 175.3, 146.7, 167.7, 184.5, 202.9, 273.3, 194.8, 187.7, 133.7, 209.1, 260.8, 211.6, 156.8, 109.2, 303.2, 240.8, 167.4, 110.4, 90.4, 162.1, 212.1, 214.8, 83.6, 182.1, 170.5, 198.2, 143.2, 184.5, 185.2, 156.5, 69.1, 139.0, 101.4, 274.3, 220.6, 107.3, 121.7, 163.5, 176.6, 118.9, 240.1, 179.3, 246.4, 177.1, 258.8, 211.8, 226.2, 220.7, 166.4, 115.1, 213.4, 155.7, 214.1, 200.4, 133.3, 155.4, 195.1, 189.8, 197.1, 217.2, 236.7, 192.8, 224.4, 114.4, 206.9, 134.7, 219.6, 183.3, 281.0, 147.9, 144.2, 175.3, 199.3, 294.9, 97.2, 74.3, 161.6, 181.5, 118.0, 238.8, 246.5, 186.5, 202.3, 201.1, 145.8, 116.7, 303.9, 107.2, 211.8, 113.7, 149.0, 118.5, 214.9, 113.9, 225.2, 172.2, 221.7, 150.0, 160.0, 142.4, 182.3, 219.6, 92.6, 238.0, 224.0, 226.0, 224.4, 204.6, 175.8, 193.7, 169.4, 252.1, 173.6, 169.1, 170.9, 230.9, 105.0, 215.6, 285.7, 198.5, 220.6, 96.7, 97.5, 235.0, 109.8, 197.7, 264.0, 129.5, 159.0, 276.2, 207.7, 234.5, 167.6, 276.7, 146.0, 220.4, 131.7, 250.3, 68.7, 207.6, 118.2, 208.8, 137.8, 209.9, 179.5, 216.0, 210.5, 234.1, 181.5, 222.5, 240.4, 109.1, 158.1, 193.0, 205.9, 198.0, 244.1, 240.7, 122.5, 111.8, 155.7, 236.6, 149.3, 181.3, 151.8, 139.9, 248.7, 61.6, 247.6, 185.9, 178.1, 80.3, 235.6, 172.4, 178.7, 225.2, 187.5, 204.4, 134.2, 262.3, 191.1, 109.6, 197.0, 228.6, 115.4, 147.2, 198.8, 129.2, 238.1, 208.0, 211.3, 194.8, 143.2, 143.7, 198.8, 179.1 ], [ 0.0, 2.6, 7.9, 17.6, 19.5, 27.0, 34.0, 37.8, 40.9, 45.0, 47.8, 49.9, 51.1, 51.8, 54.7, 55.3, 55.6, 57.5, 58.4, 58.9, 60.0, 61.3, 61.9, 62.4, 62.8, 62.9, 64.9, 67.4, 68.5, 70.7, 70.9, 72.5, 72.8, 75.8, 77.6, 78.6, 80.3, 81.3, 81.7, 82.3, 82.5, 82.6, 83.2, 83.8, 84.8, 85.7, 85.9, 86.0, 86.3, 87.2, 87.7, 88.5, 89.5, 89.7, 90.0, 91.5, 92.3, 92.8, 93.4, 93.8, 94.7, 95.0, 95.5, 95.9, 96.3, 96.8, 97.5, 98.0, 98.2, 98.4, 99.4, 99.9, 100.8, 101.1, 101.7, 102.1, 102.6, 102.8, 103.2, 103.4, 103.5, 103.7, 104.6, 104.7, 104.9, 105.0, 105.3, 105.8, 105.9, 106.4, 106.7, 107.5, 107.8, 108.3, 108.6, 109.1, 109.4, 109.5, 110.1, 110.5, 111.1, 111.6, 111.9, 112.8, 113.0, 113.3, 114.3, 114.3, 114.8, 115.0, 115.4, 115.4, 115.5, 115.5, 115.6, 115.8, 115.9, 116.2, 116.9, 117.5, 117.6, 117.9, 118.1, 118.4, 119.0, 119.2, 119.3, 119.7, 120.5, 120.7, 120.9, 121.1, 121.7, 122.0, 122.9, 123.1, 123.2, 123.7, 123.7, 124.0, 124.1, 124.3, 124.3, 124.4, 124.5, 124.7, 124.8, 125.0, 125.2, 125.4, 125.5, 125.7, 126.0, 126.1, 126.3, 126.7, 126.9, 127.3, 127.8, 128.2, 128.3, 128.5, 128.7, 128.8, 129.0, 129.3, 129.4, 129.5, 129.7, 129.9, 130.1, 130.2, 130.5, 130.9, 131.1, 131.6, 131.9, 132.0, 132.1, 132.4, 133.1, 133.3, 133.3, 133.5, 133.8, 134.0, 134.2, 134.3, 134.4, 134.7, 134.8, 134.9, 135.0, 135.1, 135.2, 135.4, 135.8, 135.9, 136.1, 136.1, 136.4, 136.7, 136.8, 137.0, 137.1, 137.4, 137.5, 137.9, 138.1, 138.3, 138.5, 138.7, 138.9, 139.0, 139.2, 139.3, 139.4, 139.6, 139.7, 139.8, 140.1, 140.1, 140.5, 140.7, 141.1, 141.2, 141.4, 141.6, 141.7, 142.0, 142.1, 142.3, 142.3, 142.4, 142.5, 142.5, 142.8, 142.9, 143.3, 143.3, 143.4, 143.5, 143.6, 143.7, 143.7, 143.9, 144.0, 144.2, 144.4, 144.5, 144.6, 144.8, 144.9, 145.0, 145.3, 145.5, 145.6, 145.9, 146.2, 146.3, 146.3, 146.3, 146.4, 146.4, 146.5, 146.6, 146.7, 146.8, 147.0, 147.0, 147.1, 147.2, 147.5, 147.7, 147.9, 148.1, 148.2, 148.2, 148.4, 148.5, 148.5, 148.6, 148.7, 149.0, 149.2, 149.3, 149.4, 149.4, 149.7, 149.7, 149.8, 149.9, 150.0, 150.1, 150.4, 150.5, 150.6, 150.7, 151.0, 151.1, 151.5, 151.5, 151.7, 151.8, 152.0, 152.1, 152.4, 152.6, 152.9, 153.1, 153.2, 153.2, 153.5, 153.5, 153.5, 153.6, 153.7, 153.8, 154.0, 154.0, 154.0, 154.1, 154.2, 154.3, 154.4, 154.5, 154.5, 154.6, 154.6, 154.7, 154.8, 155.0, 155.0, 155.2, 155.3, 155.3, 155.5, 156.0, 156.1, 156.2, 156.4, 156.6, 156.7, 157.0, 157.1, 157.2, 157.3, 157.4, 157.6, 157.6, 157.7, 157.8, 158.0, 158.0, 158.4, 158.6, 158.6, 158.7, 158.7, 158.8, 159.0, 159.1, 159.3, 159.5, 159.7, 159.7, 159.8, 159.9, 160.0, 160.1, 160.1, 160.2, 160.4, 160.4, 160.6, 160.9, 161.1, 161.3, 161.5, 161.5, 161.7, 161.9, 162.0, 162.1, 162.3, 162.4, 162.6, 162.8, 163.0, 163.3, 163.5, 163.6, 163.7, 163.8, 164.1, 164.2, 164.5, 164.8, 164.9, 165.0, 165.3, 165.4, 165.4, 165.8, 165.9, 166.0, 166.1, 166.2, 166.5, 166.5, 166.6, 166.8, 166.9, 166.9, 167.1, 167.3, 167.4, 167.5, 167.8, 167.8, 167.9, 168.0, 168.3, 168.4, 168.6, 168.6, 168.6, 168.8, 169.2, 169.2, 169.3, 169.3, 169.5, 169.5, 169.6, 169.7, 169.8, 169.9, 169.9, 170.2, 170.5, 170.5, 170.6, 170.7, 170.7, 170.9, 171.2, 171.2, 171.5, 171.6, 171.7, 171.7, 171.8, 172.1, 172.3, 172.5, 172.7, 172.8, 173.0, 173.0, 173.1, 173.2, 173.5, 173.9, 174.1, 174.3, 174.4, 174.4, 174.5, 174.5, 174.7, 174.7, 174.8, 174.9, 175.2, 175.3, 175.4, 175.4, 175.5, 175.5, 175.7, 175.7, 175.8, 175.9, 176.0, 176.2, 176.2, 176.3, 176.4, 176.6, 176.8, 176.9, 177.2, 177.2, 177.3, 177.5, 177.9, 178.2, 178.3, 178.4, 178.7, 178.8, 178.8, 179.1, 179.2, 179.2, 179.3, 179.4, 179.4, 179.5, 179.9, 180.0, 180.0, 180.5, 180.7, 180.7, 180.9, 181.1, 181.4, 181.5, 181.5, 181.6, 181.8, 182.0, 182.1, 182.1, 182.3, 182.6, 182.8, 183.0, 183.1, 183.1, 183.2, 183.3, 183.4, 183.4, 183.6, 183.8, 183.9, 184.0, 184.1, 184.2, 184.5, 184.6, 184.8, 185.0, 185.0, 185.1, 185.3, 185.3, 185.6, 186.0, 186.0, 186.1, 186.2, 186.4, 186.7, 186.8, 187.1, 187.2, 187.3, 187.4, 187.5, 187.7, 187.8, 187.8, 187.9, 188.0, 188.5, 188.8, 188.9, 189.1, 189.2, 189.3, 189.3, 189.5, 189.6, 189.8, 189.8, 190.0, 190.1, 190.2, 190.3, 190.3, 190.4, 190.4, 190.5, 190.7, 190.8, 190.9, 191.0, 191.1, 191.3, 191.3, 191.4, 191.4, 191.9, 191.9, 192.0, 192.1, 192.3, 192.3, 192.6, 192.6, 193.0, 193.0, 193.2, 193.3, 193.4, 193.6, 193.7, 193.8, 193.9, 194.0, 194.2, 194.3, 194.4, 194.6, 194.8, 194.8, 194.9, 195.0, 195.3, 195.4, 195.5, 195.7, 195.9, 196.0, 196.1, 196.1, 196.4, 196.5, 196.6, 196.6, 196.7, 196.8, 197.0, 197.1, 197.2, 197.3, 197.4, 197.6, 197.7, 197.8, 197.9, 198.1, 198.2, 198.3, 198.4, 198.4, 198.6, 198.7, 198.8, 198.9, 199.1, 199.2, 199.3, 199.6, 200.0, 200.2, 200.3, 200.3, 200.6, 201.1, 201.3, 201.3, 201.4, 201.5, 201.8, 201.9, 201.9, 202.0, 202.1, 202.4, 202.6, 202.7, 202.9, 203.1, 203.2, 203.3, 203.3, 203.4, 203.4, 203.5, 203.5, 203.7, 203.9, 204.0, 204.2, 204.4, 204.5, 204.5, 204.7, 204.9, 205.0, 205.1, 205.2, 205.2, 205.4, 205.7, 205.7, 205.9, 206.0, 206.2, 206.2, 206.2, 206.3, 206.5, 206.7, 206.9, 206.9, 207.0, 207.0, 207.2, 207.3, 207.6, 207.6, 207.7, 207.8, 207.9, 208.0, 208.3, 208.7, 208.8, 208.9, 209.1, 209.2, 209.4, 209.4, 209.7, 209.8, 209.9, 210.2, 210.3, 210.5, 210.6, 210.7, 210.8, 211.0, 211.1, 211.1, 211.2, 211.3, 211.7, 211.9, 212.0, 212.1, 212.4, 212.8, 213.0, 213.0, 213.1, 213.4, 213.5, 213.6, 213.8, 214.0, 214.1, 214.2, 214.2, 214.3, 214.3, 214.6, 214.7, 214.9, 215.1, 215.4, 215.5, 215.6, 215.8, 215.9, 215.9, 216.0, 216.0, 216.2, 216.4, 216.7, 216.8, 216.9, 217.1, 217.2, 217.2, 217.5, 217.8, 218.0, 218.5, 218.7, 218.8, 218.9, 219.2, 219.4, 219.9, 220.0, 220.1, 220.2, 220.3, 220.7, 220.8, 220.9, 221.0, 221.1, 221.1, 221.3, 221.4, 221.6, 221.8, 221.9, 222.1, 222.3, 222.4, 222.6, 222.7, 222.8, 223.0, 223.2, 223.5, 223.9, 224.0, 224.2, 224.5, 224.7, 224.9, 225.0, 225.2, 225.3, 225.5, 225.9, 226.2, 226.5, 226.9, 227.1, 227.2, 227.4, 227.4, 227.5, 227.8, 228.1, 228.4, 228.7, 228.9, 229.3, 229.6, 229.9, 230.1, 230.2, 230.2, 230.4, 230.6, 230.7, 230.9, 231.0, 231.3, 231.8, 231.9, 232.1, 232.4, 232.5, 232.6, 232.8, 233.2, 233.5, 233.8, 234.2, 234.4, 234.5, 234.8, 234.9, 235.1, 235.5, 235.6, 235.7, 236.2, 236.2, 236.4, 236.5, 236.8, 236.8, 237.1, 237.7, 237.8, 238.0, 238.9, 239.1, 239.2, 239.2, 239.5, 239.7, 239.8, 239.8, 239.9, 239.9, 240.2, 240.3, 240.8, 241.1, 241.2, 241.7, 241.8, 241.9, 242.2, 242.4, 242.5, 242.9, 243.0, 243.1, 243.4, 243.4, 243.7, 243.9, 244.3, 244.8, 244.9, 245.0, 245.2, 245.3, 245.5, 245.7, 245.8, 246.2, 246.4, 246.8, 247.2, 247.5, 247.8, 248.6, 248.9, 249.4, 249.6, 249.9, 250.9, 251.4, 251.5, 251.6, 252.3, 252.4, 252.9, 253.0, 253.2, 253.4, 254.1, 254.4, 254.7, 254.9, 255.8, 256.0, 256.5, 256.7, 257.2, 257.7, 258.0, 259.4, 259.8, 260.5, 261.3, 261.4, 261.7, 261.9, 262.2, 262.9, 263.8, 264.4, 265.1, 265.9, 266.0, 266.3, 266.7, 267.4, 268.4, 268.7, 269.0, 269.8, 270.3, 270.7, 271.1, 271.2, 271.5, 271.6, 271.8, 272.4, 272.6, 272.7, 273.2, 273.6, 274.0, 274.6, 275.2, 276.5, 277.0, 277.9, 278.5, 279.1, 279.8, 280.0, 280.4, 281.1, 281.3, 282.5, 283.2, 283.9, 285.7, 286.4, 287.1, 287.4, 288.0, 288.5, 289.5, 290.4, 291.8, 293.3, 295.0, 298.1, 299.5, 301.5, 305.1, 308.0, 309.9, 312.0, 313.2, 314.1, 315.6, 322.3, 322.4, 326.3, 335.5, 345.3 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Day Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 100.25932276039434,\n",
      "      \"sum\" : 233905.0,\n",
      "      \"std_dev\" : 20.165008436664074,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 165.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 16.5,\n",
      "            \"count\" : 2.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 16.5,\n",
      "            \"upper_bound\" : 33.0,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 33.0,\n",
      "            \"upper_bound\" : 49.5,\n",
      "            \"count\" : 14.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 49.5,\n",
      "            \"upper_bound\" : 66.0,\n",
      "            \"count\" : 80.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 66.0,\n",
      "            \"upper_bound\" : 82.5,\n",
      "            \"count\" : 344.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 82.5,\n",
      "            \"upper_bound\" : 99.0,\n",
      "            \"count\" : 636.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 99.0,\n",
      "            \"upper_bound\" : 115.5,\n",
      "            \"count\" : 737.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 115.5,\n",
      "            \"upper_bound\" : 132.0,\n",
      "            \"count\" : 377.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 132.0,\n",
      "            \"upper_bound\" : 148.5,\n",
      "            \"count\" : 127.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 148.5,\n",
      "            \"upper_bound\" : 165.0,\n",
      "            \"count\" : 16.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 110.0, 138.0, 117.0, 61.0, 112.0, 87.0, 122.0, 73.0, 86.0, 128.0, 85.0, 83.0, 108.0, 109.0, 105.0, 85.0, 104.0, 103.0, 142.0, 76.0, 94.0, 72.0, 125.0, 109.0, 148.0, 112.0, 111.0, 131.0, 130.0, 107.0, 90.0, 107.0, 121.0, 126.0, 78.0, 63.0, 103.0, 97.0, 108.0, 105.0, 96.0, 117.0, 82.0, 79.0, 106.0, 105.0, 59.0, 67.0, 104.0, 88.0, 119.0, 97.0, 87.0, 89.0, 76.0, 68.0, 77.0, 88.0, 109.0, 99.0, 88.0, 115.0, 79.0, 85.0, 121.0, 75.0, 129.0, 70.0, 115.0, 97.0, 86.0, 117.0, 92.0, 98.0, 114.0, 108.0, 106.0, 114.0, 101.0, 146.0, 85.0, 92.0, 125.0, 80.0, 100.0, 137.0, 78.0, 96.0, 91.0, 104.0, 97.0, 100.0, 66.0, 116.0, 84.0, 75.0, 127.0, 81.0, 70.0, 93.0, 96.0, 133.0, 104.0, 68.0, 103.0, 108.0, 86.0, 95.0, 87.0, 148.0, 94.0, 94.0, 107.0, 92.0, 81.0, 87.0, 102.0, 114.0, 110.0, 48.0, 110.0, 57.0, 140.0, 48.0, 136.0, 88.0, 112.0, 115.0, 93.0, 83.0, 88.0, 85.0, 84.0, 88.0, 106.0, 117.0, 89.0, 86.0, 104.0, 62.0, 87.0, 110.0, 127.0, 100.0, 122.0, 113.0, 112.0, 110.0, 104.0, 121.0, 91.0, 143.0, 96.0, 99.0, 115.0, 66.0, 109.0, 91.0, 96.0, 112.0, 106.0, 80.0, 107.0, 104.0, 95.0, 103.0, 100.0, 47.0, 94.0, 97.0, 99.0, 108.0, 92.0, 106.0, 98.0, 115.0, 67.0, 115.0, 92.0, 86.0, 102.0, 111.0, 92.0, 100.0, 106.0, 104.0, 126.0, 64.0, 126.0, 85.0, 88.0, 99.0, 127.0, 90.0, 117.0, 96.0, 91.0, 102.0, 110.0, 110.0, 105.0, 71.0, 92.0, 78.0, 113.0, 44.0, 123.0, 117.0, 97.0, 113.0, 101.0, 100.0, 68.0, 108.0, 106.0, 80.0, 95.0, 109.0, 130.0, 96.0, 121.0, 121.0, 100.0, 108.0, 100.0, 95.0, 68.0, 106.0, 101.0, 86.0, 105.0, 125.0, 85.0, 101.0, 101.0, 108.0, 74.0, 112.0, 97.0, 104.0, 99.0, 88.0, 70.0, 99.0, 82.0, 145.0, 85.0, 110.0, 69.0, 100.0, 135.0, 98.0, 117.0, 109.0, 103.0, 95.0, 95.0, 111.0, 94.0, 132.0, 114.0, 81.0, 116.0, 124.0, 88.0, 80.0, 114.0, 69.0, 108.0, 110.0, 88.0, 137.0, 121.0, 56.0, 71.0, 65.0, 125.0, 87.0, 133.0, 80.0, 114.0, 107.0, 93.0 ], [ 0.0, 35.0, 40.0, 44.0, 45.0, 49.0, 51.0, 52.0, 53.0, 54.0, 54.0, 55.0, 55.0, 55.0, 56.0, 56.0, 57.0, 57.0, 58.0, 58.0, 59.0, 59.0, 60.0, 61.0, 61.0, 61.0, 61.0, 61.0, 61.0, 62.0, 62.0, 62.0, 63.0, 63.0, 63.0, 63.0, 64.0, 65.0, 65.0, 65.0, 65.0, 65.0, 66.0, 66.0, 66.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 68.0, 68.0, 68.0, 68.0, 69.0, 69.0, 69.0, 69.0, 70.0, 70.0, 70.0, 70.0, 70.0, 70.0, 70.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 131.0, 131.0, 131.0, 131.0, 132.0, 132.0, 132.0, 132.0, 132.0, 132.0, 133.0, 133.0, 133.0, 133.0, 133.0, 133.0, 134.0, 134.0, 134.0, 134.0, 134.0, 134.0, 135.0, 135.0, 135.0, 136.0, 136.0, 137.0, 137.0, 137.0, 137.0, 137.0, 138.0, 138.0, 138.0, 138.0, 138.0, 139.0, 139.0, 139.0, 140.0, 140.0, 140.0, 140.0, 140.0, 140.0, 141.0, 141.0, 141.0, 141.0, 142.0, 142.0, 143.0, 143.0, 145.0, 145.0, 145.0, 146.0, 147.0, 147.0, 149.0, 150.0, 151.0, 151.0, 151.0, 156.0, 158.0, 163.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Eve Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 200.0501071581652,\n",
      "      \"sum\" : 466716.8999999994,\n",
      "      \"std_dev\" : 50.01592824933489,\n",
      "      \"min\" : 31.2,\n",
      "      \"max\" : 361.8,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 31.2,\n",
      "            \"upper_bound\" : 64.26,\n",
      "            \"count\" : 7.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 64.26,\n",
      "            \"upper_bound\" : 97.32000000000001,\n",
      "            \"count\" : 43.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 97.32000000000001,\n",
      "            \"upper_bound\" : 130.38,\n",
      "            \"count\" : 135.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 130.38,\n",
      "            \"upper_bound\" : 163.44,\n",
      "            \"count\" : 360.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 163.44,\n",
      "            \"upper_bound\" : 196.5,\n",
      "            \"count\" : 555.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 196.5,\n",
      "            \"upper_bound\" : 229.56,\n",
      "            \"count\" : 587.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 229.56,\n",
      "            \"upper_bound\" : 262.62,\n",
      "            \"count\" : 404.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 262.62,\n",
      "            \"upper_bound\" : 295.68,\n",
      "            \"count\" : 178.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 295.68,\n",
      "            \"upper_bound\" : 328.74,\n",
      "            \"count\" : 49.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 328.74,\n",
      "            \"upper_bound\" : 361.8,\n",
      "            \"count\" : 15.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 212.8, 221.3, 227.8, 341.3, 286.2, 158.6, 292.3, 257.5, 161.4, 102.8, 151.2, 134.0, 259.7, 127.5, 155.0, 203.9, 183.8, 189.5, 228.4, 176.6, 247.8, 194.5, 169.1, 185.3, 223.8, 290.9, 200.2, 194.8, 134.2, 127.8, 207.5, 233.9, 169.9, 248.9, 235.2, 254.9, 176.7, 204.1, 181.5, 244.4, 169.8, 221.5, 284.3, 187.1, 99.5, 253.0, 215.9, 167.2, 278.7, 137.6, 217.7, 172.2, 178.5, 211.4, 169.3, 245.0, 169.8, 348.5, 242.4, 193.5, 247.9, 199.2, 103.1, 196.7, 267.1, 253.8, 187.8, 275.0, 161.9, 221.1, 168.6, 234.7, 217.7, 282.6, 221.2, 96.6, 187.6, 191.4, 196.4, 164.6, 227.3, 249.1, 167.8, 131.1, 154.5, 264.9, 205.5, 262.3, 203.5, 246.8, 351.6, 178.6, 263.6, 209.9, 221.0, 195.3, 106.1, 163.7, 216.9, 215.8, 153.1, 170.5, 144.5, 143.8, 137.3, 276.2, 155.0, 150.1, 131.0, 120.9, 164.6, 173.7, 139.1, 209.1, 172.0, 170.4, 140.2, 230.3, 132.9, 159.1, 52.9, 211.1, 238.2, 125.8, 143.7, 162.7, 228.3, 180.4, 188.8, 256.2, 163.7, 129.5, 230.9, 140.3, 177.8, 317.0, 196.8, 204.7, 185.4, 200.9, 309.2, 185.7, 164.1, 148.8, 173.7, 259.4, 246.7, 231.9, 234.4, 147.9, 216.6, 127.8, 235.9, 210.4, 201.4, 160.6, 228.4, 226.7, 241.3, 193.4, 165.7, 186.2, 177.3, 196.3, 205.1, 167.2, 230.0, 195.5, 178.0, 69.2, 303.5, 192.2, 213.8, 232.2, 86.8, 260.5, 165.1, 245.3, 177.8, 198.2, 145.3, 184.9, 162.6, 236.1, 293.8, 189.4, 126.2, 139.8, 303.3, 177.6, 209.6, 210.7, 304.6, 159.5, 205.2, 206.6, 246.1, 184.9, 226.1, 91.7, 169.9, 201.4, 167.6, 180.6, 200.6, 167.5, 270.6, 155.2, 193.8, 268.1, 183.3, 189.6, 250.5, 132.2, 248.9, 167.9, 165.8, 164.8, 195.2, 176.0, 203.7, 203.0, 211.2, 216.5, 260.6, 209.2, 251.6, 167.2, 213.7, 286.3, 121.9, 162.3, 186.9, 250.5, 200.2, 196.9, 169.7, 201.8, 115.7, 322.2, 224.8, 209.3, 273.7, 246.9, 269.4, 273.3, 239.6, 260.3, 197.5, 200.2, 182.4, 209.9, 223.6, 220.0, 255.1, 256.3, 212.0, 236.7, 199.9, 115.9, 256.6, 233.7, 173.4, 146.6, 137.5, 165.0, 198.9, 129.2, 249.3, 222.8, 248.5, 178.7, 175.2, 230.1, 214.1, 187.2, 198.9, 165.7, 213.4, 88.1, 297.8, 195.5, 238.3 ], [ 31.2, 58.9, 60.8, 65.2, 66.5, 71.0, 73.2, 75.3, 77.1, 78.3, 79.3, 80.6, 82.2, 83.9, 87.6, 88.6, 89.7, 90.0, 90.5, 92.0, 93.7, 95.1, 98.3, 101.3, 102.2, 102.6, 103.4, 105.5, 105.7, 106.2, 106.8, 107.9, 108.2, 109.9, 110.2, 110.8, 112.5, 113.2, 113.3, 114.3, 114.5, 114.7, 115.0, 115.7, 116.5, 116.6, 117.0, 117.9, 118.0, 118.5, 118.7, 118.9, 119.3, 119.6, 120.0, 120.3, 120.4, 120.5, 120.7, 121.0, 121.6, 122.2, 122.8, 123.0, 123.4, 123.5, 123.5, 123.9, 123.9, 124.4, 126.0, 126.9, 127.3, 127.8, 128.7, 128.9, 129.1, 129.3, 129.4, 129.8, 130.1, 130.2, 130.7, 131.1, 131.4, 131.7, 131.8, 132.3, 132.5, 132.9, 133.0, 133.4, 133.9, 134.1, 134.3, 134.5, 134.7, 134.9, 135.0, 135.2, 136.0, 136.1, 136.4, 136.7, 136.9, 137.3, 137.8, 138.1, 138.3, 138.5, 138.7, 138.9, 139.1, 139.5, 139.6, 140.2, 140.9, 140.9, 141.2, 141.4, 141.6, 141.8, 141.9, 142.0, 142.1, 142.3, 142.6, 142.6, 142.7, 143.1, 143.4, 143.7, 143.8, 144.1, 144.3, 144.4, 145.0, 145.1, 145.5, 145.9, 146.4, 146.7, 146.9, 147.0, 147.4, 147.7, 148.0, 148.2, 148.3, 148.5, 148.7, 149.1, 149.3, 149.5, 149.6, 149.9, 150.0, 150.0, 150.1, 150.6, 150.8, 151.3, 151.4, 151.7, 152.0, 152.1, 152.3, 152.5, 152.5, 152.7, 152.7, 152.8, 152.9, 153.1, 153.1, 153.2, 153.3, 153.6, 153.7, 153.8, 154.0, 154.2, 154.5, 154.6, 154.7, 154.9, 154.9, 155.1, 155.4, 155.5, 155.6, 155.8, 156.0, 156.0, 156.1, 156.3, 156.4, 156.9, 157.0, 157.3, 157.5, 157.6, 157.6, 158.0, 158.2, 158.4, 158.6, 158.8, 159.2, 159.4, 159.5, 159.6, 159.6, 159.7, 159.9, 160.1, 160.1, 160.3, 160.5, 160.6, 160.6, 160.7, 160.8, 160.9, 161.1, 161.4, 161.7, 161.7, 161.7, 161.8, 161.9, 161.9, 162.1, 162.3, 162.3, 162.5, 162.5, 162.6, 162.8, 163.0, 163.1, 163.2, 163.3, 163.4, 163.6, 163.7, 164.0, 164.2, 164.5, 164.5, 164.5, 164.7, 164.9, 165.1, 165.2, 165.4, 165.6, 165.8, 165.8, 165.9, 165.9, 166.4, 166.6, 166.7, 166.8, 167.0, 167.1, 167.2, 167.2, 167.3, 167.5, 167.6, 167.7, 167.7, 167.7, 167.8, 167.9, 167.9, 168.0, 168.2, 168.3, 168.4, 168.5, 168.6, 168.7, 169.0, 169.1, 169.3, 169.5, 169.6, 169.8, 169.9, 169.9, 170.0, 170.0, 170.2, 170.5, 170.5, 170.7, 170.8, 170.9, 171.2, 171.4, 171.6, 171.7, 171.8, 171.9, 172.2, 172.7, 172.8, 173.1, 173.3, 173.3, 173.4, 173.5, 173.7, 174.0, 174.4, 174.5, 174.6, 174.8, 174.9, 175.1, 175.2, 175.4, 175.7, 175.8, 175.9, 176.0, 176.1, 176.2, 176.4, 176.6, 176.6, 176.7, 177.0, 177.2, 177.5, 177.6, 177.8, 177.9, 178.2, 178.3, 178.4, 178.6, 178.8, 178.9, 179.1, 179.3, 179.3, 179.5, 179.7, 179.9, 180.0, 180.0, 180.0, 180.2, 180.3, 180.4, 180.5, 180.5, 180.6, 180.8, 181.0, 181.2, 181.4, 181.6, 181.6, 181.7, 182.0, 182.0, 182.2, 182.2, 182.5, 182.9, 182.9, 183.0, 183.1, 183.4, 183.5, 183.6, 183.6, 183.9, 184.0, 184.1, 184.2, 184.3, 184.5, 184.6, 184.8, 185.0, 185.4, 185.5, 185.5, 185.7, 185.8, 185.9, 186.0, 186.4, 186.6, 186.6, 186.7, 186.8, 187.0, 187.0, 187.1, 187.2, 187.2, 187.3, 187.4, 187.5, 187.5, 187.7, 187.8, 188.0, 188.2, 188.2, 188.4, 188.5, 188.5, 188.6, 188.8, 188.8, 189.0, 189.1, 189.3, 189.3, 189.4, 189.6, 189.6, 189.7, 189.8, 190.0, 190.0, 190.2, 190.3, 190.4, 190.6, 190.7, 190.8, 190.9, 191.1, 191.3, 191.6, 191.8, 191.9, 192.0, 192.2, 192.3, 192.4, 192.6, 192.7, 192.8, 193.0, 193.0, 193.1, 193.3, 193.6, 193.8, 193.9, 194.0, 194.1, 194.4, 194.4, 194.5, 194.6, 194.6, 194.7, 194.8, 194.9, 195.2, 195.5, 195.5, 195.6, 195.7, 195.7, 195.7, 195.8, 195.9, 196.0, 196.2, 196.3, 196.5, 196.6, 196.7, 196.7, 196.8, 197.0, 197.0, 197.3, 197.3, 197.4, 197.4, 197.5, 197.6, 197.6, 197.7, 197.8, 198.0, 198.3, 198.4, 198.5, 198.6, 198.6, 198.9, 199.4, 199.5, 199.6, 199.7, 199.7, 199.9, 200.0, 200.2, 200.3, 200.5, 200.6, 200.7, 200.7, 200.9, 201.0, 201.0, 201.0, 201.1, 201.3, 201.6, 202.2, 202.3, 202.4, 202.4, 202.5, 202.6, 202.6, 202.7, 202.8, 202.9, 203.0, 203.2, 203.4, 203.4, 203.6, 203.7, 203.8, 203.9, 203.9, 204.0, 204.1, 204.2, 204.3, 204.5, 204.6, 204.7, 204.9, 204.9, 205.0, 205.1, 205.2, 205.2, 205.5, 205.5, 205.7, 205.9, 206.0, 206.0, 206.2, 206.4, 206.5, 206.7, 206.8, 206.9, 207.0, 207.0, 207.1, 207.3, 207.5, 207.6, 207.7, 207.9, 207.9, 208.0, 208.0, 208.2, 208.2, 208.4, 208.5, 208.6, 208.8, 208.9, 209.0, 209.1, 209.1, 209.3, 209.4, 209.4, 209.4, 209.5, 209.5, 209.6, 209.8, 209.8, 209.9, 210.0, 210.1, 210.2, 210.2, 210.4, 210.5, 210.6, 210.7, 210.8, 210.9, 211.0, 211.1, 211.2, 211.3, 211.4, 211.5, 211.5, 211.6, 211.6, 211.7, 211.8, 211.8, 212.0, 212.1, 212.2, 212.3, 212.5, 212.7, 213.0, 213.2, 213.3, 213.4, 213.6, 213.7, 213.7, 213.9, 213.9, 214.1, 214.2, 214.2, 214.3, 214.5, 214.7, 214.8, 215.1, 215.2, 215.3, 215.5, 215.5, 215.6, 215.7, 215.8, 216.2, 216.3, 216.3, 216.4, 216.5, 216.5, 216.6, 216.7, 216.8, 216.9, 217.0, 217.1, 217.2, 217.4, 217.5, 217.7, 218.2, 218.6, 218.8, 218.9, 219.1, 219.1, 219.2, 219.3, 219.5, 219.6, 219.6, 219.7, 219.9, 220.0, 220.2, 220.4, 220.5, 220.6, 220.6, 220.9, 220.9, 221.1, 221.2, 221.4, 221.6, 221.8, 221.9, 222.0, 222.1, 222.2, 222.3, 222.4, 222.7, 223.0, 223.0, 223.1, 223.2, 223.3, 223.5, 223.5, 223.5, 223.6, 224.1, 224.2, 224.3, 224.4, 224.6, 224.7, 224.9, 224.9, 225.0, 225.1, 225.1, 225.1, 225.2, 225.3, 225.5, 225.9, 225.9, 226.1, 226.2, 226.6, 226.7, 226.8, 227.2, 227.4, 227.8, 228.4, 228.6, 228.9, 229.0, 229.4, 229.4, 229.7, 229.9, 230.0, 230.1, 230.1, 230.3, 230.5, 230.7, 230.9, 230.9, 231.3, 231.4, 231.6, 231.7, 231.8, 231.8, 232.1, 232.3, 232.4, 232.6, 232.7, 232.9, 232.9, 233.2, 233.2, 233.5, 233.6, 233.7, 233.7, 233.8, 234.1, 234.5, 234.9, 235.0, 235.3, 235.3, 235.5, 235.8, 235.9, 236.0, 236.0, 236.2, 236.3, 236.5, 236.7, 236.8, 237.0, 237.2, 237.4, 237.7, 237.9, 238.1, 238.3, 238.6, 238.7, 239.5, 239.7, 240.1, 240.1, 240.2, 240.6, 240.7, 240.7, 240.8, 241.0, 241.4, 241.5, 241.6, 241.9, 242.0, 242.2, 242.2, 242.3, 242.6, 242.8, 242.8, 243.1, 243.2, 243.3, 243.5, 243.7, 243.9, 244.0, 244.3, 244.7, 244.8, 245.1, 245.4, 245.6, 246.0, 246.1, 246.3, 246.5, 246.5, 246.5, 246.6, 246.7, 247.0, 247.2, 247.3, 247.6, 247.7, 247.8, 248.1, 248.2, 248.5, 248.6, 248.7, 248.7, 248.8, 249.0, 249.1, 249.3, 249.5, 249.6, 249.7, 249.8, 249.8, 249.9, 250.0, 250.2, 250.7, 250.7, 250.8, 251.1, 251.3, 251.7, 251.8, 252.2, 252.4, 252.7, 253.0, 253.4, 253.4, 253.6, 253.6, 253.9, 254.1, 254.2, 254.5, 255.1, 255.6, 255.7, 255.9, 255.9, 256.1, 256.1, 256.6, 256.6, 256.7, 256.8, 256.8, 256.9, 257.2, 257.2, 257.5, 257.7, 258.0, 258.2, 258.4, 258.7, 258.8, 259.2, 259.3, 259.6, 259.8, 260.0, 260.2, 260.7, 260.9, 261.5, 261.5, 261.6, 262.0, 262.2, 262.3, 262.6, 263.0, 263.4, 263.7, 264.1, 264.4, 264.5, 264.7, 264.7, 265.0, 265.3, 265.5, 265.7, 265.8, 266.3, 266.5, 266.9, 267.1, 267.4, 267.8, 268.3, 268.5, 268.5, 269.1, 269.5, 269.8, 270.5, 270.8, 271.0, 271.7, 272.8, 273.0, 273.0, 273.6, 273.8, 274.0, 274.4, 274.6, 274.8, 275.0, 275.4, 275.6, 276.0, 276.3, 276.4, 276.4, 276.8, 277.1, 277.5, 278.0, 278.2, 278.5, 279.3, 280.1, 280.8, 281.1, 281.7, 282.8, 283.2, 283.3, 284.5, 285.2, 285.8, 285.9, 286.0, 286.7, 287.3, 287.4, 287.7, 288.0, 288.7, 289.4, 289.8, 290.0, 290.9, 291.3, 292.5, 292.7, 292.8, 293.6, 294.0, 295.7, 298.0, 298.6, 300.9, 301.3, 301.5, 302.6, 303.4, 303.8, 304.9, 307.2, 308.7, 313.2, 313.7, 317.2, 317.8, 318.8, 319.3, 322.3, 327.0, 328.2, 329.3, 330.6, 332.8, 337.1, 347.3, 354.2 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Eve Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 99.57393913416202,\n",
      "      \"sum\" : 232306.0,\n",
      "      \"std_dev\" : 19.67557809959058,\n",
      "      \"min\" : 12.0,\n",
      "      \"max\" : 170.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 12.0,\n",
      "            \"upper_bound\" : 27.8,\n",
      "            \"count\" : 2.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 27.8,\n",
      "            \"upper_bound\" : 43.6,\n",
      "            \"count\" : 2.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 43.6,\n",
      "            \"upper_bound\" : 59.4,\n",
      "            \"count\" : 44.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 59.4,\n",
      "            \"upper_bound\" : 75.2,\n",
      "            \"count\" : 195.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 75.2,\n",
      "            \"upper_bound\" : 91.0,\n",
      "            \"count\" : 530.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 91.0,\n",
      "            \"upper_bound\" : 106.8,\n",
      "            \"count\" : 708.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 106.8,\n",
      "            \"upper_bound\" : 122.6,\n",
      "            \"count\" : 576.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 122.6,\n",
      "            \"upper_bound\" : 138.4,\n",
      "            \"count\" : 215.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 138.4,\n",
      "            \"upper_bound\" : 154.2,\n",
      "            \"count\" : 56.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 154.2,\n",
      "            \"upper_bound\" : 170.0,\n",
      "            \"count\" : 5.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 100.0, 92.0, 128.0, 79.0, 86.0, 98.0, 112.0, 103.0, 82.0, 56.0, 82.0, 114.0, 108.0, 86.0, 101.0, 107.0, 68.0, 108.0, 139.0, 114.0, 126.0, 157.0, 126.0, 99.0, 87.0, 92.0, 64.0, 107.0, 103.0, 86.0, 109.0, 115.0, 125.0, 108.0, 100.0, 110.0, 84.0, 129.0, 109.0, 88.0, 101.0, 125.0, 119.0, 112.0, 122.0, 83.0, 116.0, 62.0, 88.0, 108.0, 109.0, 108.0, 128.0, 96.0, 60.0, 122.0, 114.0, 108.0, 106.0, 118.0, 81.0, 97.0, 94.0, 112.0, 118.0, 114.0, 121.0, 129.0, 123.0, 121.0, 116.0, 97.0, 114.0, 110.0, 82.0, 82.0, 99.0, 119.0, 77.0, 83.0, 132.0, 126.0, 90.0, 84.0, 90.0, 99.0, 89.0, 122.0, 78.0, 91.0, 80.0, 46.0, 121.0, 93.0, 147.0, 87.0, 80.0, 112.0, 80.0, 68.0, 80.0, 86.0, 92.0, 74.0, 102.0, 77.0, 86.0, 88.0, 114.0, 91.0, 59.0, 109.0, 123.0, 142.0, 103.0, 96.0, 134.0, 109.0, 93.0, 119.0, 109.0, 115.0, 133.0, 112.0, 111.0, 66.0, 97.0, 91.0, 65.0, 101.0, 108.0, 114.0, 137.0, 114.0, 118.0, 129.0, 111.0, 77.0, 118.0, 111.0, 105.0, 111.0, 45.0, 95.0, 85.0, 95.0, 89.0, 92.0, 96.0, 97.0, 123.0, 72.0, 90.0, 99.0, 87.0, 108.0, 117.0, 137.0, 146.0, 120.0, 115.0, 90.0, 116.0, 119.0, 88.0, 106.0, 121.0, 84.0, 106.0, 84.0, 74.0, 89.0, 112.0, 54.0, 122.0, 102.0, 127.0, 105.0, 109.0, 89.0, 146.0, 98.0, 76.0, 70.0, 123.0, 64.0, 118.0, 121.0, 100.0, 92.0, 84.0, 80.0, 83.0, 88.0, 94.0, 84.0, 96.0, 144.0, 103.0, 84.0, 102.0, 80.0, 121.0, 100.0, 81.0, 144.0, 74.0, 121.0, 95.0, 69.0, 79.0, 104.0, 53.0, 75.0, 90.0, 128.0, 119.0, 94.0, 116.0, 89.0, 99.0, 141.0, 79.0, 103.0, 90.0, 69.0, 123.0, 136.0, 87.0, 76.0, 105.0, 139.0, 114.0, 86.0, 121.0, 87.0, 75.0, 102.0, 96.0, 81.0, 87.0, 86.0, 121.0, 111.0, 85.0, 103.0, 102.0, 103.0, 68.0, 110.0, 108.0, 92.0, 96.0, 118.0, 110.0, 150.0, 98.0, 109.0, 124.0, 129.0, 69.0, 74.0, 88.0, 103.0, 111.0, 71.0, 96.0, 113.0, 119.0, 102.0, 109.0, 70.0, 87.0, 73.0, 68.0, 98.0, 76.0, 97.0, 73.0, 94.0, 98.0, 91.0, 102.0 ], [ 12.0, 42.0, 44.0, 48.0, 48.0, 48.0, 50.0, 52.0, 52.0, 53.0, 54.0, 56.0, 56.0, 57.0, 58.0, 58.0, 58.0, 58.0, 59.0, 59.0, 59.0, 60.0, 60.0, 60.0, 60.0, 60.0, 61.0, 61.0, 61.0, 62.0, 62.0, 63.0, 63.0, 63.0, 63.0, 63.0, 64.0, 64.0, 64.0, 65.0, 65.0, 65.0, 65.0, 65.0, 66.0, 66.0, 66.0, 66.0, 66.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 67.0, 68.0, 68.0, 68.0, 69.0, 69.0, 69.0, 69.0, 70.0, 70.0, 70.0, 70.0, 70.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 75.0, 75.0, 75.0, 75.0, 75.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 124.0, 124.0, 124.0, 124.0, 124.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 126.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 127.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 129.0, 129.0, 129.0, 129.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 132.0, 132.0, 133.0, 133.0, 133.0, 133.0, 134.0, 134.0, 134.0, 134.0, 135.0, 135.0, 136.0, 136.0, 136.0, 136.0, 136.0, 136.0, 137.0, 137.0, 137.0, 137.0, 138.0, 138.0, 138.0, 138.0, 139.0, 140.0, 140.0, 140.0, 141.0, 141.0, 141.0, 142.0, 142.0, 143.0, 143.0, 143.0, 144.0, 144.0, 146.0, 147.0, 147.0, 148.0, 149.0, 150.0, 151.0, 152.0, 154.0, 155.0, 164.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Night Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 201.3885983711959,\n",
      "      \"sum\" : 469839.6000000001,\n",
      "      \"std_dev\" : 50.6279607960497,\n",
      "      \"min\" : 23.2,\n",
      "      \"max\" : 395.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 23.2,\n",
      "            \"upper_bound\" : 60.379999999999995,\n",
      "            \"count\" : 9.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 60.379999999999995,\n",
      "            \"upper_bound\" : 97.56,\n",
      "            \"count\" : 35.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 97.56,\n",
      "            \"upper_bound\" : 134.74,\n",
      "            \"count\" : 171.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 134.74,\n",
      "            \"upper_bound\" : 171.92,\n",
      "            \"count\" : 463.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 171.92,\n",
      "            \"upper_bound\" : 209.1,\n",
      "            \"count\" : 623.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 209.1,\n",
      "            \"upper_bound\" : 246.28,\n",
      "            \"count\" : 590.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 246.28,\n",
      "            \"upper_bound\" : 283.46,\n",
      "            \"count\" : 325.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 283.46,\n",
      "            \"upper_bound\" : 320.64,\n",
      "            \"count\" : 98.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 320.64,\n",
      "            \"upper_bound\" : 357.82,\n",
      "            \"count\" : 15.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 357.82,\n",
      "            \"upper_bound\" : 395.0,\n",
      "            \"count\" : 4.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 226.3, 150.4, 214.0, 165.7, 261.7, 187.7, 201.2, 227.8, 239.6, 213.7, 191.0, 210.6, 141.5, 289.3, 189.6, 122.2, 153.8, 223.9, 197.9, 214.4, 219.0, 242.3, 221.2, 205.1, 240.3, 228.4, 233.1, 170.6, 118.9, 225.6, 169.4, 277.4, 238.8, 158.6, 206.2, 160.2, 263.4, 171.3, 281.4, 207.2, 194.1, 103.9, 305.5, 281.1, 189.5, 180.8, 216.3, 194.8, 263.4, 159.7, 152.1, 137.5, 218.3, 72.4, 141.1, 134.4, 215.8, 212.6, 231.8, 299.6, 155.1, 120.2, 211.8, 261.7, 151.0, 213.1, 154.5, 141.1, 142.1, 109.9, 139.8, 133.6, 146.9, 181.0, 131.6, 170.7, 101.7, 144.0, 120.1, 240.7, 122.4, 136.0, 163.1, 230.7, 281.4, 168.9, 213.6, 143.9, 203.4, 203.9, 215.8, 203.8, 165.2, 194.1, 145.7, 280.5, 179.6, 271.7, 153.5, 223.3, 240.0, 227.6, 125.7, 140.1, 189.6, 146.5, 189.7, 219.8, 216.9, 226.6, 128.8, 248.6, 199.1, 173.0, 183.4, 165.1, 227.4, 256.7, 272.0, 259.2, 246.1, 249.0, 271.8, 261.6, 253.4, 215.5, 180.1, 133.4, 253.2, 169.0, 242.7, 193.6, 217.1, 208.9, 206.1, 160.4, 212.4, 256.6, 192.7, 246.8, 152.1, 161.5, 157.7, 224.5, 257.1, 134.7, 226.1, 164.7, 203.2, 183.8, 250.6, 199.2, 260.2, 242.7, 177.4, 77.9, 299.7, 144.6, 211.4, 254.4, 189.2, 189.0, 296.3, 294.8, 204.0, 205.7, 206.3, 200.5, 215.6, 257.6, 224.0, 165.1, 214.3, 147.1, 156.2, 144.2, 141.5, 260.0, 255.7, 170.8, 195.2, 143.2, 250.3, 192.7, 250.7, 229.9, 274.2, 171.6, 154.5, 159.8, 233.0, 231.9, 181.2, 192.8, 164.6, 178.0, 138.0, 234.3, 155.6, 211.7, 244.9, 159.0, 270.0, 174.1, 153.8, 260.0, 209.9, 186.7, 171.7, 255.3, 139.8, 206.7, 181.2, 177.7, 268.0, 167.6, 151.6, 54.5, 268.8, 250.9, 246.2, 151.8, 259.3, 196.1, 195.0, 197.4, 191.6, 214.2, 175.1, 167.0, 253.7, 264.5, 210.7, 241.6, 237.4, 187.2, 264.3, 206.1, 295.8, 210.0, 197.6, 289.9, 217.9, 200.0, 187.1, 197.8, 268.3, 192.2, 209.5, 231.7, 180.6, 266.9, 240.8, 265.7, 225.9, 158.6, 282.3, 264.0, 170.8, 185.4, 235.3, 131.9, 145.8, 225.7, 226.0, 173.1, 165.9, 207.5, 321.2, 225.3, 167.1, 185.7, 136.3, 119.8, 214.9, 190.0, 76.4, 265.9, 190.8, 233.2, 212.6, 213.3, 165.7 ], [ 23.2, 45.0, 53.3, 56.6, 61.4, 65.8, 71.1, 75.8, 77.3, 79.3, 80.2, 82.4, 88.2, 89.7, 91.6, 94.3, 94.9, 95.3, 96.4, 97.4, 98.6, 99.3, 100.9, 102.0, 102.1, 103.7, 104.5, 104.7, 104.8, 105.4, 105.6, 107.3, 107.9, 108.1, 108.9, 109.6, 109.6, 109.6, 110.1, 110.4, 111.2, 111.6, 111.7, 112.9, 113.5, 114.2, 114.3, 114.5, 115.7, 116.1, 116.4, 117.0, 117.8, 117.9, 118.0, 118.3, 119.1, 119.1, 119.4, 120.0, 121.0, 121.1, 122.0, 122.3, 122.6, 123.0, 123.4, 124.0, 125.6, 126.3, 126.9, 127.1, 127.4, 127.7, 127.9, 128.2, 128.4, 128.7, 128.9, 129.1, 129.6, 129.6, 129.7, 129.9, 130.6, 130.9, 132.0, 132.5, 132.6, 132.9, 133.1, 133.4, 133.7, 134.0, 134.2, 134.3, 134.6, 134.9, 135.0, 135.0, 135.9, 136.1, 136.2, 136.3, 136.6, 136.7, 137.4, 138.2, 138.4, 138.7, 139.2, 139.3, 139.4, 140.0, 140.5, 140.5, 141.1, 141.5, 141.6, 141.9, 142.4, 142.6, 143.1, 143.2, 143.3, 143.6, 143.7, 143.9, 144.1, 144.2, 144.9, 145.2, 145.5, 145.7, 146.2, 146.4, 146.7, 146.8, 146.9, 147.1, 147.4, 147.7, 147.8, 148.0, 148.1, 148.2, 148.4, 148.6, 148.7, 149.3, 149.5, 149.5, 149.9, 150.0, 150.0, 150.2, 150.3, 150.4, 150.6, 150.9, 151.1, 151.2, 151.5, 151.9, 152.2, 152.3, 152.4, 152.5, 152.7, 152.8, 152.9, 153.0, 153.2, 153.4, 153.5, 153.7, 153.9, 153.9, 154.0, 154.1, 154.2, 154.4, 154.7, 154.8, 154.9, 154.9, 155.3, 155.6, 155.7, 156.3, 156.6, 156.7, 156.9, 157.0, 157.4, 157.4, 157.6, 157.8, 157.9, 158.1, 158.4, 158.5, 158.6, 158.6, 158.7, 158.8, 158.9, 159.0, 159.0, 159.4, 159.6, 159.8, 160.2, 160.5, 160.6, 160.8, 161.1, 161.3, 161.5, 161.9, 162.1, 162.2, 162.4, 162.6, 162.6, 162.8, 162.9, 163.0, 163.1, 163.2, 163.2, 163.4, 163.6, 163.7, 164.0, 164.3, 164.3, 164.4, 164.5, 164.7, 164.8, 165.1, 165.4, 165.4, 165.6, 165.7, 165.8, 165.9, 166.3, 166.4, 166.6, 166.7, 166.8, 167.0, 167.1, 167.2, 167.3, 167.5, 167.6, 167.7, 167.9, 168.2, 168.3, 168.4, 168.5, 168.6, 168.7, 168.8, 168.8, 168.9, 169.0, 169.2, 169.3, 169.4, 169.4, 169.5, 169.6, 169.7, 169.7, 169.9, 170.1, 170.2, 170.2, 170.3, 170.5, 170.5, 170.6, 170.8, 170.9, 170.9, 171.0, 171.0, 171.4, 171.5, 171.7, 172.0, 172.4, 172.5, 172.7, 172.7, 172.8, 172.9, 172.9, 173.1, 173.2, 173.4, 173.6, 173.8, 173.9, 174.0, 174.1, 174.3, 174.4, 174.7, 174.9, 175.0, 175.1, 175.3, 175.4, 175.8, 175.8, 175.9, 175.9, 176.0, 176.0, 176.1, 176.3, 176.4, 176.6, 176.9, 177.1, 177.3, 177.4, 177.5, 177.7, 177.9, 178.1, 178.2, 178.3, 178.4, 178.5, 178.6, 178.7, 179.1, 179.3, 179.3, 179.4, 179.5, 179.6, 179.7, 179.7, 179.8, 179.8, 180.0, 180.1, 180.2, 180.5, 180.5, 180.6, 180.6, 180.9, 180.9, 181.1, 181.2, 181.2, 181.4, 181.5, 181.5, 181.7, 181.8, 182.1, 182.1, 182.2, 182.2, 182.3, 182.4, 182.4, 182.5, 182.5, 182.7, 183.4, 183.5, 183.5, 183.9, 184.0, 184.2, 184.3, 184.4, 184.5, 184.6, 184.8, 184.9, 185.2, 185.3, 185.4, 185.5, 185.6, 185.7, 185.8, 185.9, 186.1, 186.2, 186.4, 186.5, 186.6, 186.7, 186.9, 187.2, 187.3, 187.4, 187.5, 187.9, 187.9, 188.1, 188.2, 188.2, 188.2, 188.3, 188.4, 188.7, 188.8, 188.9, 189.0, 189.1, 189.3, 189.3, 189.4, 189.4, 189.6, 189.6, 189.8, 190.1, 190.4, 190.5, 190.5, 190.6, 190.6, 190.7, 190.8, 190.9, 191.0, 191.4, 191.4, 191.4, 191.6, 191.9, 192.0, 192.0, 192.1, 192.4, 192.5, 192.6, 192.7, 193.0, 193.0, 193.1, 193.4, 193.4, 193.6, 193.6, 193.7, 193.8, 193.8, 194.0, 194.1, 194.3, 194.3, 194.3, 194.4, 194.7, 194.7, 195.0, 195.1, 195.3, 195.4, 195.5, 195.6, 195.9, 196.0, 196.2, 196.2, 196.3, 196.8, 196.8, 196.9, 197.1, 197.3, 197.4, 197.4, 197.4, 197.6, 197.7, 197.8, 197.9, 198.0, 198.4, 198.5, 198.5, 198.7, 198.9, 198.9, 199.4, 199.5, 199.7, 199.8, 200.0, 200.1, 200.4, 200.4, 200.7, 200.8, 200.9, 201.0, 201.1, 201.4, 201.4, 201.6, 201.7, 201.8, 202.0, 202.0, 202.5, 202.7, 203.0, 203.0, 203.1, 203.1, 203.3, 203.5, 203.6, 203.7, 203.7, 203.8, 203.9, 204.0, 204.0, 204.1, 204.3, 204.4, 204.6, 204.6, 204.7, 204.8, 205.0, 205.0, 205.1, 205.5, 205.7, 205.7, 205.9, 206.1, 206.1, 206.1, 206.2, 206.4, 206.4, 206.6, 206.7, 206.9, 207.1, 207.2, 207.4, 207.8, 207.9, 208.0, 208.0, 208.2, 208.3, 208.4, 208.4, 208.7, 208.8, 209.0, 209.3, 209.5, 209.6, 210.0, 210.0, 210.1, 210.1, 210.2, 210.5, 210.5, 210.5, 210.6, 210.6, 210.7, 210.8, 210.8, 211.3, 211.4, 211.7, 211.9, 212.1, 212.4, 212.5, 212.6, 212.7, 212.8, 212.8, 212.9, 213.1, 213.3, 213.5, 213.7, 213.7, 213.8, 213.9, 214.0, 214.0, 214.1, 214.2, 214.2, 214.5, 214.5, 214.6, 214.6, 214.6, 214.7, 214.7, 214.7, 214.8, 215.3, 215.4, 215.5, 215.6, 215.7, 215.8, 215.9, 216.0, 216.1, 216.4, 216.5, 216.6, 216.8, 217.0, 217.0, 217.2, 217.4, 217.5, 217.5, 217.6, 217.8, 218.0, 218.3, 218.4, 218.5, 218.7, 218.8, 219.3, 219.4, 219.6, 219.7, 220.0, 220.0, 220.2, 220.2, 220.3, 220.4, 220.6, 220.8, 220.9, 221.0, 221.3, 221.5, 221.6, 221.6, 221.7, 221.7, 222.1, 222.2, 222.4, 222.6, 222.7, 222.8, 223.0, 223.1, 223.5, 223.5, 223.7, 223.8, 224.0, 224.2, 224.2, 224.3, 224.6, 224.7, 224.8, 224.9, 225.1, 225.2, 225.5, 225.6, 225.6, 226.0, 226.1, 226.5, 226.5, 226.7, 226.7, 226.9, 227.0, 227.1, 227.2, 227.3, 227.4, 227.5, 227.6, 227.7, 228.0, 228.1, 228.2, 228.3, 228.4, 228.4, 228.5, 228.6, 228.6, 228.7, 228.9, 229.0, 229.2, 229.4, 229.5, 229.5, 229.6, 229.8, 229.9, 229.9, 230.1, 230.1, 230.1, 230.5, 230.6, 230.7, 230.8, 231.1, 231.2, 231.4, 231.5, 231.5, 231.5, 231.9, 232.1, 232.5, 232.7, 232.8, 233.0, 233.1, 233.2, 233.3, 233.4, 233.6, 233.7, 234.0, 234.5, 234.7, 234.8, 234.9, 234.9, 235.0, 235.3, 235.4, 235.5, 235.7, 236.0, 236.0, 236.1, 236.3, 236.4, 236.5, 236.7, 236.9, 237.0, 237.3, 237.6, 237.7, 238.2, 238.4, 238.4, 238.6, 238.9, 239.2, 239.4, 239.5, 239.7, 239.8, 239.9, 239.9, 240.1, 240.2, 240.3, 240.3, 240.4, 240.6, 240.7, 241.0, 241.1, 241.2, 241.3, 241.4, 241.5, 242.0, 242.6, 242.7, 242.8, 242.9, 243.1, 243.2, 243.3, 243.3, 243.5, 243.6, 244.0, 244.3, 244.3, 244.4, 244.7, 244.7, 244.9, 245.0, 245.0, 245.1, 245.3, 245.3, 245.5, 245.5, 245.7, 245.9, 245.9, 246.0, 246.1, 246.5, 246.8, 247.0, 247.1, 247.2, 247.5, 247.8, 247.9, 248.1, 248.3, 249.0, 249.4, 249.4, 249.7, 249.7, 249.9, 250.1, 250.5, 250.6, 250.8, 251.4, 251.6, 251.9, 252.1, 252.3, 252.5, 252.9, 253.0, 253.2, 253.4, 253.5, 253.9, 254.0, 254.2, 254.4, 254.7, 254.8, 254.9, 255.1, 255.2, 255.3, 255.6, 255.7, 255.8, 255.9, 256.1, 256.3, 256.5, 256.6, 256.7, 256.7, 257.0, 257.5, 257.9, 258.2, 258.2, 258.3, 258.6, 259.0, 259.0, 259.2, 259.6, 259.8, 259.9, 260.1, 260.4, 260.6, 260.8, 261.3, 261.6, 261.8, 262.0, 262.2, 262.4, 262.7, 262.9, 263.2, 263.3, 263.6, 263.7, 263.9, 263.9, 264.1, 264.2, 264.7, 264.8, 265.0, 265.3, 265.5, 266.0, 266.7, 267.4, 267.6, 268.1, 268.2, 268.2, 268.5, 268.7, 269.0, 269.2, 269.3, 269.7, 269.8, 270.0, 270.1, 270.2, 270.6, 270.9, 271.5, 271.8, 271.8, 271.9, 272.1, 272.8, 273.1, 273.4, 274.0, 274.7, 275.0, 275.8, 276.6, 276.9, 277.6, 277.8, 278.4, 279.1, 279.2, 279.6, 279.8, 279.8, 280.2, 280.5, 280.8, 281.1, 281.5, 281.9, 282.5, 282.8, 283.2, 284.4, 284.6, 285.1, 285.3, 285.4, 285.9, 286.5, 286.7, 286.9, 287.6, 287.7, 288.0, 288.8, 289.2, 289.3, 289.5, 289.6, 290.0, 291.0, 291.6, 291.8, 292.1, 293.3, 293.9, 294.8, 297.1, 298.9, 299.0, 300.0, 302.0, 302.8, 304.2, 304.3, 305.4, 306.2, 306.6, 307.6, 309.2, 310.5, 311.1, 311.6, 312.1, 312.8, 314.1, 316.7, 320.7, 323.0, 326.4, 329.2, 333.5, 345.8, 352.2, 364.9, 381.9 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Night Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 100.22717531075868,\n",
      "      \"sum\" : 233830.0,\n",
      "      \"std_dev\" : 19.28202933863294,\n",
      "      \"min\" : 42.0,\n",
      "      \"max\" : 175.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 42.0,\n",
      "            \"upper_bound\" : 55.3,\n",
      "            \"count\" : 20.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 55.3,\n",
      "            \"upper_bound\" : 68.6,\n",
      "            \"count\" : 101.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 68.6,\n",
      "            \"upper_bound\" : 81.9,\n",
      "            \"count\" : 281.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 81.9,\n",
      "            \"upper_bound\" : 95.2,\n",
      "            \"count\" : 540.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 95.2,\n",
      "            \"upper_bound\" : 108.5,\n",
      "            \"count\" : 604.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 108.5,\n",
      "            \"upper_bound\" : 121.8,\n",
      "            \"count\" : 493.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 121.8,\n",
      "            \"upper_bound\" : 135.1,\n",
      "            \"count\" : 212.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 135.1,\n",
      "            \"upper_bound\" : 148.4,\n",
      "            \"count\" : 66.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 148.4,\n",
      "            \"upper_bound\" : 161.7,\n",
      "            \"count\" : 15.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 161.7,\n",
      "            \"upper_bound\" : 175.0,\n",
      "            \"count\" : 1.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 123.0, 120.0, 101.0, 97.0, 129.0, 87.0, 112.0, 119.0, 76.0, 84.0, 131.0, 113.0, 111.0, 83.0, 84.0, 78.0, 67.0, 93.0, 61.0, 91.0, 78.0, 138.0, 104.0, 82.0, 96.0, 91.0, 96.0, 93.0, 105.0, 86.0, 96.0, 94.0, 112.0, 88.0, 107.0, 115.0, 81.0, 84.0, 56.0, 97.0, 63.0, 89.0, 101.0, 112.0, 75.0, 123.0, 106.0, 98.0, 175.0, 121.0, 76.0, 101.0, 107.0, 84.0, 99.0, 121.0, 77.0, 118.0, 78.0, 109.0, 108.0, 113.0, 96.0, 83.0, 114.0, 125.0, 109.0, 92.0, 103.0, 100.0, 87.0, 82.0, 123.0, 98.0, 103.0, 145.0, 107.0, 78.0, 133.0, 106.0, 96.0, 73.0, 107.0, 67.0, 107.0, 108.0, 124.0, 76.0, 110.0, 117.0, 90.0, 116.0, 84.0, 100.0, 110.0, 89.0, 90.0, 117.0, 60.0, 77.0, 102.0, 80.0, 98.0, 111.0, 105.0, 111.0, 87.0, 111.0, 104.0, 110.0, 102.0, 75.0, 139.0, 96.0, 96.0, 104.0, 111.0, 96.0, 120.0, 53.0, 119.0, 129.0, 116.0, 122.0, 82.0, 98.0, 111.0, 122.0, 88.0, 151.0, 72.0, 106.0, 99.0, 110.0, 102.0, 121.0, 98.0, 101.0, 116.0, 126.0, 118.0, 113.0, 128.0, 117.0, 84.0, 135.0, 89.0, 85.0, 101.0, 74.0, 102.0, 120.0, 113.0, 88.0, 84.0, 74.0, 90.0, 72.0, 109.0, 117.0, 63.0, 92.0, 90.0, 111.0, 82.0, 102.0, 66.0, 96.0, 100.0, 64.0, 119.0, 96.0, 112.0, 76.0, 117.0, 96.0, 142.0, 94.0, 98.0, 139.0, 137.0, 146.0, 101.0, 91.0, 65.0, 118.0, 71.0, 96.0, 65.0, 72.0, 95.0, 75.0, 132.0, 74.0, 84.0, 105.0, 92.0, 89.0, 83.0, 103.0, 106.0, 124.0, 87.0, 115.0, 107.0, 97.0, 130.0, 89.0, 88.0, 62.0, 106.0, 85.0, 67.0, 91.0, 115.0, 101.0, 79.0, 91.0, 94.0, 113.0, 88.0, 120.0, 112.0, 126.0, 104.0, 42.0, 100.0, 106.0, 86.0, 77.0, 104.0, 133.0, 109.0, 125.0, 89.0, 119.0, 94.0, 112.0, 84.0, 96.0, 91.0, 84.0, 71.0, 133.0, 74.0, 71.0, 81.0, 98.0, 102.0, 101.0, 103.0, 86.0, 93.0, 78.0, 96.0, 72.0, 81.0, 118.0, 117.0, 136.0, 104.0, 120.0, 99.0, 129.0, 100.0, 102.0, 90.0, 117.0, 101.0, 91.0, 124.0, 113.0, 80.0, 81.0, 100.0, 115.0, 97.0, 72.0, 92.0, 135.0, 86.0, 120.0, 96.0 ], [ 44.0, 49.0, 50.0, 51.0, 52.0, 53.0, 53.0, 54.0, 55.0, 57.0, 57.0, 57.0, 58.0, 58.0, 59.0, 59.0, 59.0, 60.0, 60.0, 60.0, 61.0, 61.0, 61.0, 62.0, 62.0, 63.0, 63.0, 63.0, 63.0, 64.0, 64.0, 64.0, 64.0, 65.0, 65.0, 65.0, 66.0, 66.0, 66.0, 66.0, 67.0, 67.0, 67.0, 67.0, 67.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 68.0, 69.0, 69.0, 69.0, 69.0, 69.0, 70.0, 70.0, 70.0, 70.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 71.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 72.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 73.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 74.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 75.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 76.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 77.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 78.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 79.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 80.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 81.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 82.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 83.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 84.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 85.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 86.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 87.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 88.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 89.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 90.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 91.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 93.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 94.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 95.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 96.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 97.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 98.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 99.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 101.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 102.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 103.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 104.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 105.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 106.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 107.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 108.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 109.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 110.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 111.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 112.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 113.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 114.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 115.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 116.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 117.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 118.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 119.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 120.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 121.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 122.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 123.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 124.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 125.0, 126.0, 126.0, 126.0, 126.0, 126.0, 127.0, 127.0, 127.0, 127.0, 127.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 128.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 129.0, 130.0, 130.0, 130.0, 130.0, 130.0, 130.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 131.0, 132.0, 132.0, 132.0, 132.0, 132.0, 133.0, 133.0, 133.0, 134.0, 134.0, 134.0, 134.0, 135.0, 135.0, 136.0, 136.0, 136.0, 136.0, 137.0, 137.0, 137.0, 137.0, 138.0, 138.0, 138.0, 138.0, 139.0, 139.0, 140.0, 140.0, 140.0, 141.0, 142.0, 142.0, 143.0, 144.0, 144.0, 145.0, 145.0, 146.0, 147.0, 148.0, 148.0, 151.0, 151.0, 153.0, 154.0, 155.0, 156.0, 157.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Intl Mins\",\n",
      "    \"inferred_type\" : \"Fractional\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 10.253064723531933,\n",
      "      \"sum\" : 23920.4,\n",
      "      \"std_dev\" : 2.7787657409289586,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 18.4,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 1.8399999999999999,\n",
      "            \"count\" : 15.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 1.8399999999999999,\n",
      "            \"upper_bound\" : 3.6799999999999997,\n",
      "            \"count\" : 15.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 3.6799999999999997,\n",
      "            \"upper_bound\" : 5.52,\n",
      "            \"count\" : 79.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 5.52,\n",
      "            \"upper_bound\" : 7.359999999999999,\n",
      "            \"count\" : 226.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 7.359999999999999,\n",
      "            \"upper_bound\" : 9.2,\n",
      "            \"count\" : 427.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 9.2,\n",
      "            \"upper_bound\" : 11.04,\n",
      "            \"count\" : 644.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 11.04,\n",
      "            \"upper_bound\" : 12.879999999999999,\n",
      "            \"count\" : 529.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 12.879999999999999,\n",
      "            \"upper_bound\" : 14.719999999999999,\n",
      "            \"count\" : 296.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 14.719999999999999,\n",
      "            \"upper_bound\" : 16.56,\n",
      "            \"count\" : 82.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 16.56,\n",
      "            \"upper_bound\" : 18.4,\n",
      "            \"count\" : 20.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 10.0, 11.2, 9.3, 6.3, 11.3, 10.5, 0.0, 9.7, 8.2, 10.5, 8.5, 11.4, 9.7, 14.5, 8.0, 14.6, 11.0, 7.4, 8.4, 8.8, 11.3, 14.2, 10.4, 4.1, 15.4, 13.9, 8.0, 8.6, 9.4, 9.9, 5.6, 9.2, 8.2, 14.4, 8.0, 17.2, 14.2, 12.3, 6.7, 11.6, 11.9, 11.9, 11.3, 12.9, 13.4, 8.7, 16.9, 9.7, 5.9, 11.0, 6.5, 7.1, 8.0, 11.0, 8.0, 8.4, 7.6, 7.5, 11.6, 10.8, 11.9, 18.0, 7.1, 6.8, 13.0, 8.9, 12.6, 11.2, 7.2, 12.4, 9.4, 10.8, 10.9, 11.4, 6.8, 7.9, 10.5, 10.0, 9.7, 10.6, 8.5, 10.8, 10.5, 7.6, 17.3, 15.4, 12.2, 5.6, 13.7, 7.5, 8.7, 12.8, 12.0, 12.8, 10.0, 5.9, 14.0, 17.0, 7.8, 7.6, 9.8, 11.5, 11.6, 10.3, 7.7, 12.7, 11.0, 7.7, 9.4, 10.7, 12.7, 11.3, 8.8, 11.9, 13.7, 9.5, 12.2, 6.5, 12.1, 12.2, 10.9, 6.8, 10.0, 8.3, 12.6, 14.6, 8.2, 8.0, 12.1, 3.8, 7.4, 10.9, 10.7, 6.4, 12.4, 10.0, 11.4, 5.7, 8.2, 9.2, 10.0, 5.6, 9.0, 6.7, 10.3, 14.6, 15.8, 12.7, 13.0, 6.7, 11.0, 9.2, 7.6, 13.8, 10.4, 0.0, 9.6, 13.8, 7.8, 7.0, 9.8, 10.4, 14.5, 13.8, 14.7, 11.8, 13.2, 11.7, 12.1, 6.7, 13.2, 9.9, 9.7, 5.8, 9.7, 10.8, 10.8, 8.3, 12.1, 8.2, 11.8, 9.9, 8.7, 8.0, 10.3, 10.4, 4.6, 11.6, 9.7, 14.4, 10.0, 2.1, 12.6, 13.0, 10.7, 11.1, 14.6, 2.0, 13.8, 9.7, 9.9, 4.1, 7.6, 10.2, 12.4, 8.7, 8.1, 10.5, 9.7, 13.2, 5.7, 11.1, 10.5, 10.6, 11.9, 12.3, 2.2, 7.9, 11.4, 13.4, 8.3, 13.3, 13.6, 11.0, 13.3, 11.4, 10.9, 12.2, 12.4, 14.1, 9.6, 6.6, 4.9, 11.5, 13.1, 10.3, 9.0, 16.1, 8.3, 8.9, 10.3, 14.5, 7.6, 7.2, 10.1, 8.0, 6.9, 11.0, 9.5, 11.9, 6.7, 11.9, 12.7, 13.2, 12.4, 10.8, 11.3, 8.4, 16.6, 16.2, 0.0, 9.1, 11.7, 6.4, 10.0, 10.7, 6.6, 12.9, 8.3, 10.6, 9.0, 6.0, 13.3, 9.9, 10.3, 11.8, 8.6, 13.3, 11.5, 8.8, 11.4, 16.5, 10.6 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.1, 2.0, 2.5, 2.9, 3.3, 3.5, 3.6, 3.7, 3.8, 3.8, 4.1, 4.2, 4.2, 4.3, 4.3, 4.4, 4.5, 4.6, 4.7, 4.7, 4.7, 4.8, 4.8, 4.9, 5.0, 5.0, 5.0, 5.1, 5.1, 5.1, 5.2, 5.3, 5.3, 5.3, 5.3, 5.3, 5.4, 5.4, 5.4, 5.4, 5.5, 5.5, 5.5, 5.5, 5.6, 5.6, 5.6, 5.7, 5.7, 5.8, 5.8, 5.8, 5.8, 5.9, 5.9, 5.9, 5.9, 5.9, 5.9, 5.9, 6.0, 6.0, 6.0, 6.1, 6.1, 6.1, 6.1, 6.2, 6.2, 6.3, 6.3, 6.3, 6.3, 6.3, 6.3, 6.4, 6.4, 6.4, 6.4, 6.4, 6.4, 6.4, 6.5, 6.5, 6.5, 6.5, 6.5, 6.5, 6.6, 6.6, 6.6, 6.6, 6.6, 6.6, 6.6, 6.7, 6.7, 6.7, 6.7, 6.7, 6.8, 6.8, 6.8, 6.8, 6.8, 6.8, 6.9, 6.9, 6.9, 6.9, 6.9, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.1, 7.1, 7.1, 7.1, 7.1, 7.1, 7.1, 7.2, 7.2, 7.2, 7.2, 7.2, 7.2, 7.2, 7.3, 7.3, 7.3, 7.3, 7.3, 7.3, 7.3, 7.3, 7.3, 7.4, 7.4, 7.4, 7.4, 7.4, 7.4, 7.4, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.5, 7.6, 7.6, 7.6, 7.6, 7.6, 7.6, 7.7, 7.7, 7.7, 7.7, 7.7, 7.7, 7.7, 7.7, 7.7, 7.7, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.8, 7.9, 7.9, 7.9, 7.9, 7.9, 7.9, 7.9, 7.9, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.1, 8.1, 8.1, 8.1, 8.1, 8.1, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.2, 8.3, 8.3, 8.3, 8.3, 8.3, 8.3, 8.3, 8.3, 8.3, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.4, 8.5, 8.5, 8.5, 8.5, 8.5, 8.5, 8.5, 8.5, 8.5, 8.5, 8.6, 8.6, 8.6, 8.6, 8.6, 8.6, 8.6, 8.6, 8.6, 8.6, 8.7, 8.7, 8.7, 8.7, 8.7, 8.7, 8.7, 8.7, 8.7, 8.7, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.8, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 8.9, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.1, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.2, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.3, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.4, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.5, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.6, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.7, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.8, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 9.9, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.1, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.2, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.4, 10.4, 10.4, 10.4, 10.4, 10.4, 10.4, 10.4, 10.4, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.5, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.6, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.7, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.8, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 10.9, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.1, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.2, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.3, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.4, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.5, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.6, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.7, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.8, 11.9, 11.9, 11.9, 11.9, 11.9, 11.9, 11.9, 11.9, 11.9, 11.9, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.1, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.2, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.3, 12.4, 12.4, 12.4, 12.4, 12.4, 12.4, 12.4, 12.4, 12.4, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.5, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.6, 12.7, 12.7, 12.7, 12.7, 12.7, 12.7, 12.7, 12.7, 12.7, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.8, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 12.9, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.1, 13.2, 13.2, 13.2, 13.2, 13.2, 13.2, 13.2, 13.2, 13.2, 13.2, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.3, 13.4, 13.4, 13.4, 13.4, 13.4, 13.4, 13.4, 13.4, 13.4, 13.4, 13.5, 13.5, 13.5, 13.5, 13.5, 13.5, 13.5, 13.5, 13.6, 13.6, 13.6, 13.6, 13.7, 13.7, 13.7, 13.7, 13.7, 13.8, 13.8, 13.8, 13.8, 13.8, 13.8, 13.8, 13.9, 13.9, 13.9, 13.9, 13.9, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.1, 14.1, 14.1, 14.1, 14.1, 14.1, 14.2, 14.2, 14.2, 14.2, 14.3, 14.3, 14.3, 14.3, 14.3, 14.4, 14.4, 14.4, 14.4, 14.5, 14.5, 14.5, 14.6, 14.6, 14.6, 14.7, 14.7, 14.7, 14.7, 14.8, 14.8, 14.8, 14.8, 14.9, 14.9, 15.0, 15.0, 15.0, 15.1, 15.1, 15.1, 15.2, 15.2, 15.3, 15.3, 15.4, 15.4, 15.5, 15.5, 15.6, 15.6, 15.6, 15.6, 15.7, 15.7, 15.8, 15.8, 15.9, 15.9, 16.0, 16.1, 16.2, 16.3, 16.4, 16.4, 16.5, 16.5, 16.7, 16.7, 16.9, 17.3, 17.5, 17.6, 18.2 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"Intl Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 4.4950707243891985,\n",
      "      \"sum\" : 10487.0,\n",
      "      \"std_dev\" : 2.4881367315548273,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 20.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 2.0,\n",
      "            \"count\" : 124.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 2.0,\n",
      "            \"upper_bound\" : 4.0,\n",
      "            \"count\" : 805.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 4.0,\n",
      "            \"upper_bound\" : 6.0,\n",
      "            \"count\" : 765.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 6.0,\n",
      "            \"upper_bound\" : 8.0,\n",
      "            \"count\" : 383.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 8.0,\n",
      "            \"upper_bound\" : 10.0,\n",
      "            \"count\" : 157.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 10.0,\n",
      "            \"upper_bound\" : 12.0,\n",
      "            \"count\" : 62.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 12.0,\n",
      "            \"upper_bound\" : 14.0,\n",
      "            \"count\" : 25.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 14.0,\n",
      "            \"upper_bound\" : 16.0,\n",
      "            \"count\" : 6.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 16.0,\n",
      "            \"upper_bound\" : 18.0,\n",
      "            \"count\" : 4.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 18.0,\n",
      "            \"upper_bound\" : 20.0,\n",
      "            \"count\" : 2.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 6.0, 2.0, 11.0, 3.0, 3.0, 6.0, 0.0, 13.0, 3.0, 2.0, 2.0, 2.0, 2.0, 4.0, 5.0, 15.0, 9.0, 5.0, 9.0, 5.0, 5.0, 3.0, 8.0, 2.0, 8.0, 5.0, 5.0, 4.0, 6.0, 4.0, 5.0, 4.0, 5.0, 2.0, 13.0, 9.0, 4.0, 5.0, 5.0, 4.0, 3.0, 6.0, 2.0, 3.0, 3.0, 6.0, 4.0, 6.0, 2.0, 5.0, 5.0, 5.0, 3.0, 4.0, 1.0, 3.0, 4.0, 7.0, 4.0, 3.0, 4.0, 5.0, 6.0, 3.0, 2.0, 1.0, 5.0, 5.0, 6.0, 4.0, 6.0, 7.0, 2.0, 4.0, 3.0, 5.0, 6.0, 11.0, 4.0, 2.0, 2.0, 2.0, 8.0, 5.0, 3.0, 4.0, 6.0, 11.0, 3.0, 11.0, 4.0, 3.0, 7.0, 3.0, 4.0, 2.0, 6.0, 6.0, 1.0, 6.0, 5.0, 3.0, 1.0, 3.0, 6.0, 2.0, 9.0, 2.0, 3.0, 9.0, 4.0, 2.0, 1.0, 9.0, 3.0, 13.0, 2.0, 4.0, 1.0, 2.0, 5.0, 3.0, 3.0, 2.0, 5.0, 6.0, 5.0, 6.0, 5.0, 4.0, 2.0, 7.0, 9.0, 2.0, 9.0, 2.0, 3.0, 4.0, 2.0, 6.0, 2.0, 4.0, 3.0, 2.0, 1.0, 5.0, 7.0, 6.0, 3.0, 2.0, 3.0, 1.0, 6.0, 8.0, 15.0, 0.0, 9.0, 4.0, 2.0, 10.0, 5.0, 6.0, 3.0, 2.0, 9.0, 2.0, 8.0, 4.0, 4.0, 3.0, 2.0, 2.0, 6.0, 3.0, 4.0, 7.0, 3.0, 3.0, 4.0, 5.0, 9.0, 1.0, 4.0, 4.0, 7.0, 7.0, 4.0, 7.0, 5.0, 4.0, 5.0, 5.0, 4.0, 2.0, 5.0, 2.0, 3.0, 7.0, 3.0, 7.0, 10.0, 5.0, 4.0, 2.0, 6.0, 4.0, 10.0, 11.0, 3.0, 4.0, 7.0, 3.0, 3.0, 8.0, 3.0, 5.0, 4.0, 3.0, 4.0, 6.0, 3.0, 2.0, 8.0, 5.0, 2.0, 4.0, 6.0, 3.0, 6.0, 3.0, 4.0, 2.0, 10.0, 2.0, 9.0, 2.0, 3.0, 6.0, 6.0, 6.0, 8.0, 4.0, 4.0, 2.0, 3.0, 3.0, 4.0, 1.0, 10.0, 3.0, 2.0, 5.0, 4.0, 2.0, 5.0, 6.0, 4.0, 2.0, 3.0, 2.0, 0.0, 4.0, 4.0, 6.0, 4.0, 5.0, 5.0, 1.0, 4.0, 6.0, 1.0, 3.0, 3.0, 3.0, 4.0, 4.0, 6.0, 6.0, 7.0, 7.0, 8.0, 7.0, 1.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 13.0, 13.0, 13.0, 13.0, 13.0, 14.0, 15.0, 16.0, 17.0, 19.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"CustServ Calls\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 1.5803686240891557,\n",
      "      \"sum\" : 3687.0,\n",
      "      \"std_dev\" : 1.307464947005547,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 8.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 482.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 1.6,\n",
      "            \"count\" : 804.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 1.6,\n",
      "            \"upper_bound\" : 2.4,\n",
      "            \"count\" : 551.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 2.4,\n",
      "            \"upper_bound\" : 3.2,\n",
      "            \"count\" : 312.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 3.2,\n",
      "            \"upper_bound\" : 4.0,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 4.0,\n",
      "            \"upper_bound\" : 4.8,\n",
      "            \"count\" : 115.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 4.8,\n",
      "            \"upper_bound\" : 5.6,\n",
      "            \"count\" : 45.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 5.6,\n",
      "            \"upper_bound\" : 6.4,\n",
      "            \"count\" : 17.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 6.4,\n",
      "            \"upper_bound\" : 7.2,\n",
      "            \"count\" : 6.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 7.2,\n",
      "            \"upper_bound\" : 8.0,\n",
      "            \"count\" : 1.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 1.0, 0.0, 0.0, 2.0, 3.0, 2.0, 3.0, 2.0, 4.0, 4.0, 1.0, 0.0, 2.0, 3.0, 4.0, 0.0, 2.0, 2.0, 2.0, 2.0, 5.0, 2.0, 8.0, 0.0, 3.0, 1.0, 1.0, 1.0, 0.0, 3.0, 2.0, 0.0, 0.0, 4.0, 3.0, 2.0, 0.0, 3.0, 3.0, 3.0, 4.0, 2.0, 1.0, 1.0, 1.0, 3.0, 0.0, 1.0, 2.0, 2.0, 1.0, 0.0, 4.0, 3.0, 1.0, 3.0, 1.0, 3.0, 0.0, 3.0, 0.0, 1.0, 0.0, 1.0, 1.0, 2.0, 1.0, 1.0, 3.0, 2.0, 1.0, 1.0, 3.0, 1.0, 1.0, 1.0, 3.0, 2.0, 4.0, 1.0, 2.0, 2.0, 2.0, 1.0, 2.0, 2.0, 0.0, 1.0, 1.0, 1.0, 1.0, 5.0, 1.0, 0.0, 3.0, 0.0, 0.0, 1.0, 1.0, 1.0, 2.0, 0.0, 0.0, 0.0, 2.0, 1.0, 5.0, 3.0, 3.0, 0.0, 3.0, 1.0, 2.0, 1.0, 3.0, 1.0, 2.0, 0.0, 0.0, 1.0, 0.0, 1.0, 4.0, 6.0, 1.0, 1.0, 2.0, 3.0, 1.0, 0.0, 0.0, 2.0, 3.0, 0.0, 1.0, 1.0, 2.0, 1.0, 3.0, 0.0, 1.0, 2.0, 0.0, 0.0, 0.0, 2.0, 3.0, 1.0, 3.0, 2.0, 0.0, 3.0, 3.0, 2.0, 3.0, 1.0, 3.0, 3.0, 0.0, 0.0, 3.0, 2.0, 2.0, 1.0, 2.0, 2.0, 1.0, 1.0, 2.0, 1.0, 1.0, 1.0, 2.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 3.0, 1.0, 1.0, 2.0, 1.0, 3.0, 2.0, 0.0, 2.0, 1.0, 0.0, 2.0, 2.0, 1.0, 1.0, 3.0, 3.0, 2.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0, 2.0, 3.0, 2.0, 1.0, 3.0, 3.0, 1.0, 1.0, 3.0, 0.0, 2.0, 2.0, 0.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 3.0, 3.0, 2.0, 1.0, 1.0, 2.0, 1.0, 2.0, 1.0, 2.0, 0.0, 0.0, 1.0, 2.0, 2.0, 1.0, 0.0, 0.0, 2.0, 1.0, 1.0, 2.0, 2.0, 0.0, 1.0, 2.0, 1.0, 1.0, 2.0, 3.0, 0.0, 0.0, 0.0, 2.0, 1.0, 0.0, 4.0, 0.0, 0.0, 3.0, 0.0, 4.0, 2.0, 3.0, 3.0, 2.0, 2.0, 5.0, 4.0, 0.0, 1.0, 0.0, 4.0, 4.0, 5.0, 2.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 7.0, 7.0, 7.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AK\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.014573510501500214,\n",
      "      \"sum\" : 34.0,\n",
      "      \"std_dev\" : 0.11983790424220074,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2299.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 34.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.021431633090441493,\n",
      "      \"sum\" : 50.0,\n",
      "      \"std_dev\" : 0.14481822465946118,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2283.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 50.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AR\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.018859837119588514,\n",
      "      \"sum\" : 44.0,\n",
      "      \"std_dev\" : 0.13602993664414878,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2289.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 44.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_AZ\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.020574367766823833,\n",
      "      \"sum\" : 48.0,\n",
      "      \"std_dev\" : 0.14195444043008756,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2286.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 47.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.009858551221603087,\n",
      "      \"sum\" : 23.0,\n",
      "      \"std_dev\" : 0.09879959609944836,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2310.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 23.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CO\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.02100300042863266,\n",
      "      \"sum\" : 49.0,\n",
      "      \"std_dev\" : 0.143394122618842,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2284.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 49.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_CT\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.021431633090441493,\n",
      "      \"sum\" : 50.0,\n",
      "      \"std_dev\" : 0.14481822465946126,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2284.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 49.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_DC\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.015859408486926702,\n",
      "      \"sum\" : 37.0,\n",
      "      \"std_dev\" : 0.12493153184593325,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2297.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 36.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_DE\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.018859837119588514,\n",
      "      \"sum\" : 44.0,\n",
      "      \"std_dev\" : 0.13602993664414878,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2290.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 43.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_FL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.018431204457779682,\n",
      "      \"sum\" : 43.0,\n",
      "      \"std_dev\" : 0.1345046287679914,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2290.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 43.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_GA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.015430775825117874,\n",
      "      \"sum\" : 36.0,\n",
      "      \"std_dev\" : 0.12325853715890364,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2298.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 35.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_HI\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.016716673810544362,\n",
      "      \"sum\" : 39.0,\n",
      "      \"std_dev\" : 0.12820774792209796,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2294.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 39.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.014144877839691384,\n",
      "      \"sum\" : 33.0,\n",
      "      \"std_dev\" : 0.11808810384874349,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2300.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 33.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_ID\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.022288898414059153,\n",
      "      \"sum\" : 52.0,\n",
      "      \"std_dev\" : 0.1476214869913827,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2282.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 51.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IL\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.016288041148735534,\n",
      "      \"sum\" : 38.0,\n",
      "      \"std_dev\" : 0.12658096564757523,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2295.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 38.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_IN\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.020145735105015,\n",
      "      \"sum\" : 47.0,\n",
      "      \"std_dev\" : 0.14049869914733581,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2286.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 47.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_KS\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.02186026575225032,\n",
      "      \"sum\" : 51.0,\n",
      "      \"std_dev\" : 0.14622720175634665,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2282.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 51.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "            \"data\" : [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 ] ]\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_KY\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.018859837119588514,\n",
      "      \"sum\" : 44.0,\n",
      "      \"std_dev\" : 0.13602993664414845,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2289.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 44.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "[TRUNCATED MESSAGE] 6571 bytes are truncated.\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_LA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.017573939134162022,\n",
      "      \"sum\" : 41.0,\n",
      "      \"std_dev\" : 0.13139671151695853,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2292.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 41.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "[TRUNCATED MESSAGE] 6571 bytes are truncated.\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MA\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.015430775825117874,\n",
      "      \"sum\" : 36.0,\n",
      "      \"std_dev\" : 0.12325853715890375,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2298.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 35.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "[TRUNCATED MESSAGE] 6571 bytes are truncated.\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_MD\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.020574367766823833,\n",
      "      \"sum\" : 48.0,\n",
      "      \"std_dev\" : 0.1419544404300876,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "            \"count\" : 2285.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.1,\n",
      "            \"upper_bound\" : 0.2,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.2,\n",
      "            \"upper_bound\" : 0.3,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.3,\n",
      "            \"upper_bound\" : 0.4,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.4,\n",
      "            \"upper_bound\" : 0.5,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.5,\n",
      "            \"upper_bound\" : 0.6,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.6,\n",
      "            \"upper_bound\" : 0.7,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.7,\n",
      "            \"upper_bound\" : 0.8,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.8,\n",
      "            \"upper_bound\" : 0.9,\n",
      "            \"count\" : 0.0\n",
      "          }, {\n",
      "            \"lower_bound\" : 0.9,\n",
      "            \"upper_bound\" : 1.0,\n",
      "            \"count\" : 48.0\n",
      "          } ],\n",
      "          \"sketch\" : {\n",
      "            \"parameters\" : {\n",
      "              \"c\" : 0.64,\n",
      "              \"k\" : 2048.0\n",
      "            },\n",
      "[TRUNCATED MESSAGE] 6571 bytes are truncated.\n",
      "          }\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }, {\n",
      "    \"name\" : \"State_ME\",\n",
      "    \"inferred_type\" : \"Integral\",\n",
      "    \"numerical_statistics\" : {\n",
      "      \"common\" : {\n",
      "        \"num_present\" : 2333,\n",
      "        \"num_missing\" : 0\n",
      "      },\n",
      "      \"mean\" : 0.017573939134162022,\n",
      "      \"sum\" : 41.0,\n",
      "      \"std_dev\" : 0.1313967115169584,\n",
      "      \"min\" : 0.0,\n",
      "      \"max\" : 1.0,\n",
      "      \"distribution\" : {\n",
      "        \"kll\" : {\n",
      "          \"buckets\" : [ {\n",
      "            \"lower_bound\" : 0.0,\n",
      "            \"upper_bound\" : 0.1,\n",
      "          }\u001b[0m\n",
      "\u001b[34m}\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  FileUtil:29 - Write to file statistics.json at path /opt/ml/processing/output.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  YarnClientSchedulerBackend:54 - Interrupting monitor thread\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  YarnClientSchedulerBackend:54 - Shutting down all executors\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  YarnSchedulerBackend$YarnDriverEndpoint:54 - Asking each executor to shut down\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  SchedulerExtensionServices:54 - Stopping SchedulerExtensionServices\u001b[0m\n",
      "\u001b[34m(serviceOption=None,\n",
      " services=List(),\n",
      " started=false)\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  YarnClientSchedulerBackend:54 - Stopped\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  MemoryStore:54 - MemoryStore cleared\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  BlockManager:54 - BlockManager stopped\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  BlockManagerMaster:54 - BlockManagerMaster stopped\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  SparkContext:54 - Successfully stopped SparkContext\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  Main:65 - Completed: Job completed successfully with no violations.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  Main:141 - Write to file /opt/ml/output/message.\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  ShutdownHookManager:54 - Shutdown hook called\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-08634062-0e5b-4863-94ab-afa99cfb1e91\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-b51fc04a-9ef0-4ddc-9c9a-1fd6503f2b3f\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27,859 - DefaultDataAnalyzer - INFO - Completed spark-submit with return code : 0\u001b[0m\n",
      "\u001b[34m2021-11-08 13:23:27,859 - DefaultDataAnalyzer - INFO - Spark job completed.\u001b[0m\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sagemaker.processing.ProcessingJob at 0x7fce67e0a410>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sagemaker.model_monitor import DefaultModelMonitor\n",
    "from sagemaker.model_monitor.dataset_format import DatasetFormat\n",
    "\n",
    "my_default_monitor = DefaultModelMonitor(\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    volume_size_in_gb=20,\n",
    "    max_runtime_in_seconds=3600,\n",
    ")\n",
    "\n",
    "my_default_monitor.suggest_baseline(\n",
    "    baseline_dataset=baseline_data_uri + \"/training-dataset-with-header.csv\",\n",
    "    dataset_format=DatasetFormat.csv(header=True),\n",
    "    output_s3_uri=baseline_results_uri,\n",
    "    wait=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 생성된 제약 조건 및 통계 탐색"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found Files:\n",
      "sagemaker/DEMO-ModelMonitor/baselining/results/constraints.json\n",
      " sagemaker/DEMO-ModelMonitor/baselining/results/statistics.json\n"
     ]
    }
   ],
   "source": [
    "s3_client = boto3.Session().client(\"s3\")\n",
    "result = s3_client.list_objects(Bucket=bucket, Prefix=baseline_results_prefix)\n",
    "report_files = [report_file.get(\"Key\") for report_file in result.get(\"Contents\")]\n",
    "print(\"Found Files:\")\n",
    "print(\"\\n \".join(report_files))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:4: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>inferred_type</th>\n",
       "      <th>numerical_statistics.common.num_present</th>\n",
       "      <th>numerical_statistics.common.num_missing</th>\n",
       "      <th>numerical_statistics.mean</th>\n",
       "      <th>numerical_statistics.sum</th>\n",
       "      <th>numerical_statistics.std_dev</th>\n",
       "      <th>numerical_statistics.min</th>\n",
       "      <th>numerical_statistics.max</th>\n",
       "      <th>numerical_statistics.distribution.kll.buckets</th>\n",
       "      <th>numerical_statistics.distribution.kll.sketch.parameters.c</th>\n",
       "      <th>numerical_statistics.distribution.kll.sketch.parameters.k</th>\n",
       "      <th>numerical_statistics.distribution.kll.sketch.data</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Churn</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>0.139306</td>\n",
       "      <td>325.0</td>\n",
       "      <td>0.346265</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'lower_bound': 0.0, 'upper_bound': 0.1, 'cou...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Account Length</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>101.276897</td>\n",
       "      <td>236279.0</td>\n",
       "      <td>39.552442</td>\n",
       "      <td>1.0</td>\n",
       "      <td>243.0</td>\n",
       "      <td>[{'lower_bound': 1.0, 'upper_bound': 25.2, 'co...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[119.0, 100.0, 111.0, 181.0, 95.0, 104.0, 70....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>VMail Message</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>8.214316</td>\n",
       "      <td>19164.0</td>\n",
       "      <td>13.776908</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>[{'lower_bound': 0.0, 'upper_bound': 5.1, 'cou...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[19.0, 0.0, 0.0, 40.0, 36.0, 0.0, 0.0, 24.0, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Day Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>180.226489</td>\n",
       "      <td>420468.4</td>\n",
       "      <td>53.987179</td>\n",
       "      <td>0.0</td>\n",
       "      <td>350.8</td>\n",
       "      <td>[{'lower_bound': 0.0, 'upper_bound': 35.08, 'c...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[178.1, 160.3, 197.1, 105.2, 283.1, 113.6, 23...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Day Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>100.259323</td>\n",
       "      <td>233905.0</td>\n",
       "      <td>20.165008</td>\n",
       "      <td>0.0</td>\n",
       "      <td>165.0</td>\n",
       "      <td>[{'lower_bound': 0.0, 'upper_bound': 16.5, 'co...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[110.0, 138.0, 117.0, 61.0, 112.0, 87.0, 122....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Eve Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>200.050107</td>\n",
       "      <td>466716.9</td>\n",
       "      <td>50.015928</td>\n",
       "      <td>31.2</td>\n",
       "      <td>361.8</td>\n",
       "      <td>[{'lower_bound': 31.2, 'upper_bound': 64.26, '...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[212.8, 221.3, 227.8, 341.3, 286.2, 158.6, 29...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Eve Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>99.573939</td>\n",
       "      <td>232306.0</td>\n",
       "      <td>19.675578</td>\n",
       "      <td>12.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>[{'lower_bound': 12.0, 'upper_bound': 27.8, 'c...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[100.0, 92.0, 128.0, 79.0, 86.0, 98.0, 112.0,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Night Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>201.388598</td>\n",
       "      <td>469839.6</td>\n",
       "      <td>50.627961</td>\n",
       "      <td>23.2</td>\n",
       "      <td>395.0</td>\n",
       "      <td>[{'lower_bound': 23.2, 'upper_bound': 60.37999...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[226.3, 150.4, 214.0, 165.7, 261.7, 187.7, 20...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Night Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>100.227175</td>\n",
       "      <td>233830.0</td>\n",
       "      <td>19.282029</td>\n",
       "      <td>42.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>[{'lower_bound': 42.0, 'upper_bound': 55.3, 'c...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[123.0, 120.0, 101.0, 97.0, 129.0, 87.0, 112....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Intl Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>2333</td>\n",
       "      <td>0</td>\n",
       "      <td>10.253065</td>\n",
       "      <td>23920.4</td>\n",
       "      <td>2.778766</td>\n",
       "      <td>0.0</td>\n",
       "      <td>18.4</td>\n",
       "      <td>[{'lower_bound': 0.0, 'upper_bound': 1.8399999...</td>\n",
       "      <td>0.64</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>[[10.0, 11.2, 9.3, 6.3, 11.3, 10.5, 0.0, 9.7, ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             name inferred_type  numerical_statistics.common.num_present  \\\n",
       "0           Churn      Integral                                     2333   \n",
       "1  Account Length      Integral                                     2333   \n",
       "2   VMail Message      Integral                                     2333   \n",
       "3        Day Mins    Fractional                                     2333   \n",
       "4       Day Calls      Integral                                     2333   \n",
       "5        Eve Mins    Fractional                                     2333   \n",
       "6       Eve Calls      Integral                                     2333   \n",
       "7      Night Mins    Fractional                                     2333   \n",
       "8     Night Calls      Integral                                     2333   \n",
       "9       Intl Mins    Fractional                                     2333   \n",
       "\n",
       "   numerical_statistics.common.num_missing  numerical_statistics.mean  \\\n",
       "0                                        0                   0.139306   \n",
       "1                                        0                 101.276897   \n",
       "2                                        0                   8.214316   \n",
       "3                                        0                 180.226489   \n",
       "4                                        0                 100.259323   \n",
       "5                                        0                 200.050107   \n",
       "6                                        0                  99.573939   \n",
       "7                                        0                 201.388598   \n",
       "8                                        0                 100.227175   \n",
       "9                                        0                  10.253065   \n",
       "\n",
       "   numerical_statistics.sum  numerical_statistics.std_dev  \\\n",
       "0                     325.0                      0.346265   \n",
       "1                  236279.0                     39.552442   \n",
       "2                   19164.0                     13.776908   \n",
       "3                  420468.4                     53.987179   \n",
       "4                  233905.0                     20.165008   \n",
       "5                  466716.9                     50.015928   \n",
       "6                  232306.0                     19.675578   \n",
       "7                  469839.6                     50.627961   \n",
       "8                  233830.0                     19.282029   \n",
       "9                   23920.4                      2.778766   \n",
       "\n",
       "   numerical_statistics.min  numerical_statistics.max  \\\n",
       "0                       0.0                       1.0   \n",
       "1                       1.0                     243.0   \n",
       "2                       0.0                      51.0   \n",
       "3                       0.0                     350.8   \n",
       "4                       0.0                     165.0   \n",
       "5                      31.2                     361.8   \n",
       "6                      12.0                     170.0   \n",
       "7                      23.2                     395.0   \n",
       "8                      42.0                     175.0   \n",
       "9                       0.0                      18.4   \n",
       "\n",
       "       numerical_statistics.distribution.kll.buckets  \\\n",
       "0  [{'lower_bound': 0.0, 'upper_bound': 0.1, 'cou...   \n",
       "1  [{'lower_bound': 1.0, 'upper_bound': 25.2, 'co...   \n",
       "2  [{'lower_bound': 0.0, 'upper_bound': 5.1, 'cou...   \n",
       "3  [{'lower_bound': 0.0, 'upper_bound': 35.08, 'c...   \n",
       "4  [{'lower_bound': 0.0, 'upper_bound': 16.5, 'co...   \n",
       "5  [{'lower_bound': 31.2, 'upper_bound': 64.26, '...   \n",
       "6  [{'lower_bound': 12.0, 'upper_bound': 27.8, 'c...   \n",
       "7  [{'lower_bound': 23.2, 'upper_bound': 60.37999...   \n",
       "8  [{'lower_bound': 42.0, 'upper_bound': 55.3, 'c...   \n",
       "9  [{'lower_bound': 0.0, 'upper_bound': 1.8399999...   \n",
       "\n",
       "   numerical_statistics.distribution.kll.sketch.parameters.c  \\\n",
       "0                                               0.64           \n",
       "1                                               0.64           \n",
       "2                                               0.64           \n",
       "3                                               0.64           \n",
       "4                                               0.64           \n",
       "5                                               0.64           \n",
       "6                                               0.64           \n",
       "7                                               0.64           \n",
       "8                                               0.64           \n",
       "9                                               0.64           \n",
       "\n",
       "   numerical_statistics.distribution.kll.sketch.parameters.k  \\\n",
       "0                                             2048.0           \n",
       "1                                             2048.0           \n",
       "2                                             2048.0           \n",
       "3                                             2048.0           \n",
       "4                                             2048.0           \n",
       "5                                             2048.0           \n",
       "6                                             2048.0           \n",
       "7                                             2048.0           \n",
       "8                                             2048.0           \n",
       "9                                             2048.0           \n",
       "\n",
       "   numerical_statistics.distribution.kll.sketch.data  \n",
       "0  [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0,...  \n",
       "1  [[119.0, 100.0, 111.0, 181.0, 95.0, 104.0, 70....  \n",
       "2  [[19.0, 0.0, 0.0, 40.0, 36.0, 0.0, 0.0, 24.0, ...  \n",
       "3  [[178.1, 160.3, 197.1, 105.2, 283.1, 113.6, 23...  \n",
       "4  [[110.0, 138.0, 117.0, 61.0, 112.0, 87.0, 122....  \n",
       "5  [[212.8, 221.3, 227.8, 341.3, 286.2, 158.6, 29...  \n",
       "6  [[100.0, 92.0, 128.0, 79.0, 86.0, 98.0, 112.0,...  \n",
       "7  [[226.3, 150.4, 214.0, 165.7, 261.7, 187.7, 20...  \n",
       "8  [[123.0, 120.0, 101.0, 97.0, 129.0, 87.0, 112....  \n",
       "9  [[10.0, 11.2, 9.3, 6.3, 11.3, 10.5, 0.0, 9.7, ...  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "baseline_job = my_default_monitor.latest_baselining_job\n",
    "schema_df = pd.io.json.json_normalize(baseline_job.baseline_statistics().body_dict[\"features\"])\n",
    "schema_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:2: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>inferred_type</th>\n",
       "      <th>completeness</th>\n",
       "      <th>num_constraints.is_non_negative</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Churn</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Account Length</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>VMail Message</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Day Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Day Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Eve Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Eve Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Night Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Night Calls</td>\n",
       "      <td>Integral</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Intl Mins</td>\n",
       "      <td>Fractional</td>\n",
       "      <td>1.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             name inferred_type  completeness  num_constraints.is_non_negative\n",
       "0           Churn      Integral           1.0                             True\n",
       "1  Account Length      Integral           1.0                             True\n",
       "2   VMail Message      Integral           1.0                             True\n",
       "3        Day Mins    Fractional           1.0                             True\n",
       "4       Day Calls      Integral           1.0                             True\n",
       "5        Eve Mins    Fractional           1.0                             True\n",
       "6       Eve Calls      Integral           1.0                             True\n",
       "7      Night Mins    Fractional           1.0                             True\n",
       "8     Night Calls      Integral           1.0                             True\n",
       "9       Intl Mins    Fractional           1.0                             True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "constraints_df = pd.io.json.json_normalize(\n",
    "    baseline_job.suggested_constraints().body_dict[\"features\"]\n",
    ")\n",
    "constraints_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 데이터 품질 모니터링을 위해 수집 데이터 분석하기\n",
    "\n",
    "이제 데이터를 수집작업에 대한 모니터링 스케줄을 설정합니다. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 스케줄 생성"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, copy over some test scripts to the S3 bucket so that they can be used for pre and post processing\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(code_prefix + \"/preprocessor.py\").upload_file(\n",
    "    \"preprocessor.py\"\n",
    ")\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(code_prefix + \"/postprocessor.py\").upload_file(\n",
    "    \"postprocessor.py\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "앞서 생성한 엔드포인트에 대한 모델 모니터링 스케줄을 생성할 수 있습니다. 기준선 정보(제약 조건 및 통계)를 사용하여 실시간 추론 트래픽과 비교합니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The endpoint attribute has been renamed in sagemaker>=2.\n",
      "See: https://sagemaker.readthedocs.io/en/stable/v2.html for details.\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.model_monitor import CronExpressionGenerator\n",
    "from time import gmtime, strftime\n",
    "\n",
    "mon_schedule_name = \"DEMO-xgb-churn-pred-model-monitor-schedule-\" + strftime(\n",
    "    \"%Y-%m-%d-%H-%M-%S\", gmtime()\n",
    ")\n",
    "my_default_monitor.create_monitoring_schedule(\n",
    "    monitor_schedule_name=mon_schedule_name,\n",
    "    endpoint_input=predictor.endpoint,\n",
    "    # record_preprocessor_script=pre_processor_script,\n",
    "    post_analytics_processor_script=s3_code_postprocessor_uri,\n",
    "    output_s3_uri=s3_report_path,\n",
    "    statistics=my_default_monitor.baseline_statistics(),\n",
    "    constraints=my_default_monitor.suggested_constraints(),\n",
    "    schedule_cron_expression=CronExpressionGenerator.hourly(),\n",
    "    enable_cloudwatch_metrics=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 인공 트래픽 생성 시작\n",
    "\n",
    "아래 셀은 일부 트래픽을 엔드포인트로 보내기 위해 스레드를 시작합니다. 이 스레드를 종료하려면 커널을 중지해야 합니다. 트래픽이 없으면 처리할 데이터가 없기 때문에 모니터링 작업은 실패(`Failed`)로 표시됩니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The endpoint attribute has been renamed in sagemaker>=2.\n",
      "See: https://sagemaker.readthedocs.io/en/stable/v2.html for details.\n"
     ]
    }
   ],
   "source": [
    "from threading import Thread\n",
    "from time import sleep\n",
    "import time\n",
    "\n",
    "endpoint_name = predictor.endpoint\n",
    "runtime_client = boto3.client(\"runtime.sagemaker\")\n",
    "\n",
    "# (just repeating code from above for convenience/ able to run this section independently)\n",
    "def invoke_endpoint(ep_name, file_name, runtime_client):\n",
    "    with open(file_name, \"r\") as f:\n",
    "        for row in f:\n",
    "            payload = row.rstrip(\"\\n\")\n",
    "            response = runtime_client.invoke_endpoint(\n",
    "                EndpointName=ep_name, ContentType=\"text/csv\", Body=payload\n",
    "            )\n",
    "            response[\"Body\"].read()\n",
    "            time.sleep(1)\n",
    "\n",
    "\n",
    "def invoke_endpoint_forever():\n",
    "    while True:\n",
    "        invoke_endpoint(endpoint_name, \"test_data/test-dataset-input-cols.csv\", runtime_client)\n",
    "\n",
    "\n",
    "thread = Thread(target=invoke_endpoint_forever)\n",
    "thread.start()\n",
    "\n",
    "# Note that you need to stop the kernel to stop the invocations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 생성된 스케줄 살펴보기 \n",
    "\n",
    "`MonitoringScheduleStatus`가 Scheduled로 변경되는 것을 확인합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Schedule status: Scheduled\n"
     ]
    }
   ],
   "source": [
    "desc_schedule_result = my_default_monitor.describe_schedule()\n",
    "print(\"Schedule status: {}\".format(desc_schedule_result[\"MonitoringScheduleStatus\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 실행 이력 살펴보기\n",
    "\n",
    "스케줄 생성시 지정한 시간 간격으로 분석 작업이 진행됩니다. 여기에 최근 실행결과가 리스트업됩니다. 우리는 1시간 간격으로 스케줄을 만들었으므로 스케줄 생성한 이후단,  바로 이 작업을 시작하는 경우 실행이 비어 있을 수 있습니다. 실행이 시작되는 것을 보려면 시간 경계(UTC 기준)를 넘을 때까지 기다려야 할 수도 있습니다. 아래 코드에 대기 로직이 있습니다.\n",
    "\n",
    "참고: 시간별 일정의 경우에도 Amazon SageMaker에는 실행까지 20분의 버퍼 기간이 있습니다. 그래서 매 시간 경계에서 0분에서 ~20분 사이에 실행이 시작되는 것을 볼 수 있습니다. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No executions found for schedule. monitoring_schedule_name: DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22\n",
      "We created a hourly schedule above and it will kick off executions ON the hour (plus 0 - 20 min buffer.\n",
      "We will have to wait till we hit the hour...\n",
      "Waiting for the 1st execution to happen...\n",
      "No executions found for schedule. monitoring_schedule_name: DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22\n",
      "Waiting for the 1st execution to happen...\n"
     ]
    }
   ],
   "source": [
    "mon_executions = my_default_monitor.list_executions()\n",
    "print(\n",
    "    \"We created a hourly schedule above and it will kick off executions ON the hour (plus 0 - 20 min buffer.\\nWe will have to wait till we hit the hour...\"\n",
    ")\n",
    "\n",
    "while len(mon_executions) == 0:\n",
    "    print(\"Waiting for the 1st execution to happen...\")\n",
    "    time.sleep(60)\n",
    "    mon_executions = my_default_monitor.list_executions()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 특정 실행결과 (최근 실행) 살펴보기 \n",
    "\n",
    "이전 셀에서는 가장 최근에 완료 또는 실패한 스케줄 실행을 살펴보았습니다. 여기서 가능한 터미널 상태와 각각의 의미는 다음과 같습니다.\n",
    "\n",
    "* Completed - 모니터링 실행이 완료되었으며 위반 보고서에서 문제가 발견되지 않았음을 의미합니다.\n",
    "* CompletedWithViolations - 실행이 완료되었지만 제약 조건 위반이 감지되었음을 의미합니다.\n",
    "* Failed - 클라이언트 오류(아마도 잘못된 역할 권한) 또는 인프라 문제로 인해 모니터링 실행이 실패했습니다. 정확히 무슨 일이 일어났는지 확인하려면 FailureReason 및 ExitMessage에 대한 추가 조사가 필요합니다.\n",
    "* Stopped - 작업이 최대 런타임을 초과했거나 수동으로 중지되었습니다.\n",
    "     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!Latest execution status: Completed\n",
      "Latest execution result: CompletedWithViolations: Job completed successfully with 60 violations.\n"
     ]
    }
   ],
   "source": [
    "latest_execution = mon_executions[\n",
    "    -1\n",
    "]  # latest execution's index is -1, second to last is -2 and so on..\n",
    "time.sleep(60)\n",
    "latest_execution.wait(logs=False)\n",
    "\n",
    "print(\"Latest execution status: {}\".format(latest_execution.describe()[\"ProcessingJobStatus\"]))\n",
    "print(\"Latest execution result: {}\".format(latest_execution.describe()[\"ExitMessage\"]))\n",
    "\n",
    "latest_job = latest_execution.describe()\n",
    "if latest_job[\"ProcessingJobStatus\"] != \"Completed\":\n",
    "    print(\n",
    "        \"====STOP==== \\n No completed executions to inspect further. Please wait till an execution completes or investigate previously reported failures.\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Report Uri: s3://sagemaker-ap-northeast-2-095389425207/sagemaker/DEMO-ModelMonitor/reports/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22/2021/11/08/14\n"
     ]
    }
   ],
   "source": [
    "report_uri = latest_execution.output.destination\n",
    "print(\"Report Uri: {}\".format(report_uri))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 생성된 보고서 리스트업"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Report bucket: sagemaker-ap-northeast-2-095389425207\n",
      "Report key: sagemaker/DEMO-ModelMonitor/reports/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22/2021/11/08/14\n",
      "Found Report Files:\n",
      "sagemaker/DEMO-ModelMonitor/reports/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22/2021/11/08/14/constraint_violations.json\n",
      " sagemaker/DEMO-ModelMonitor/reports/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22/2021/11/08/14/constraints.json\n",
      " sagemaker/DEMO-ModelMonitor/reports/DEMO-xgb-churn-pred-model-monitor-2021-11-08-12-23-43/DEMO-xgb-churn-pred-model-monitor-schedule-2021-11-08-13-37-22/2021/11/08/14/statistics.json\n"
     ]
    }
   ],
   "source": [
    "from urllib.parse import urlparse\n",
    "\n",
    "s3uri = urlparse(report_uri)\n",
    "report_bucket = s3uri.netloc\n",
    "report_key = s3uri.path.lstrip(\"/\")\n",
    "print(\"Report bucket: {}\".format(report_bucket))\n",
    "print(\"Report key: {}\".format(report_key))\n",
    "\n",
    "s3_client = boto3.Session().client(\"s3\")\n",
    "result = s3_client.list_objects(Bucket=report_bucket, Prefix=report_key)\n",
    "report_files = [report_file.get(\"Key\") for report_file in result.get(\"Contents\")]\n",
    "print(\"Found Report Files:\")\n",
    "print(\"\\n \".join(report_files))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 위반 보고서"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "기준과 비교하여 위반 사항이 있는 경우 여기에 나열됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:2: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n",
      "  \n",
      "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:3: FutureWarning: pandas.io.json.json_normalize is deprecated, use pandas.json_normalize instead\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature_name</th>\n",
       "      <th>constraint_check_type</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>State_MD</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>State_TX</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>State_MA</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>State_RI</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>State_UT</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>State_CT</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>State_MT</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>State_PA</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>State_WA</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Churn</td>\n",
       "      <td>data_type_check</td>\n",
       "      <td>Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 0.0% of data is Integral.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  feature_name constraint_check_type  \\\n",
       "0  State_MD     data_type_check        \n",
       "1  State_TX     data_type_check        \n",
       "2  State_MA     data_type_check        \n",
       "3  State_RI     data_type_check        \n",
       "4  State_UT     data_type_check        \n",
       "5  State_CT     data_type_check        \n",
       "6  State_MT     data_type_check        \n",
       "7  State_PA     data_type_check        \n",
       "8  State_WA     data_type_check        \n",
       "9  Churn        data_type_check        \n",
       "\n",
       "                                                                                                                                            description  \n",
       "0  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "1  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "2  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "3  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "4  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "5  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "6  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "7  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "8  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 99.63768115942028% of data is Integral.  \n",
       "9  Data type match requirement is not met. Expected data type: Integral, Expected match: 100.0%. Observed: Only 0.0% of data is Integral.                "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "violations = my_default_monitor.latest_monitoring_constraint_violations()\n",
    "pd.set_option(\"display.max_colwidth\", -1)\n",
    "constraints_df = pd.io.json.json_normalize(violations.body_dict[\"violations\"])\n",
    "constraints_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 스케줄 시작/종료\n",
    "\n",
    "다음 코드를 이용하여 모니터링 스케줄을 중지하거나 시작할 수 있습니다.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# my_default_monitor.stop_monitoring_schedule()\n",
    "# my_default_monitor.start_monitoring_schedule()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 리소스 삭제\n",
    "\n",
    "엔드포인트를 계속 실행하여 데이터 캡처를 계속할 수 있습니다. 더 많은 데이터를 수집하거나 이 엔드포인트를 더 이상 사용할 계획이 없는 경우 추가 요금이 발생하지 않도록 엔드포인트를 삭제해야 합니다. 엔드포인트를 삭제해도 모델 호출 중에 캡처된 데이터는 삭제되지 않습니다. 해당 데이터는 사용자가 직접 삭제할 때까지 Amazon S3에 유지됩니다.\n",
    "\n",
    "엔드포인트를 삭제하기 위해서는 그 전에 먼저 스케줄을 삭제해야 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_default_monitor.delete_monitoring_schedule()\n",
    "time.sleep(60)  # actually wait for the deletion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:ap-northeast-2:806072073708:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}