{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Simple Hello World example for PyWren\n", "\n", "## Setup Logging (optional)\n", "Only activate the below lines if you want to see all debug messages from PyWren." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import logging\n", "logger = logging.getLogger()\n", "logger.setLevel(logging.INFO)\n", "%env PYWREN_LOGLEVEL=INFO" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hello World Examples" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "This is a simple Hello World example, showing how to take a function and run it with pywren. First we import the necessary libraries. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "import pywren\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "Pywren is designed to run any existing python functions you have, in parallel, at scale, on the cloud. So first, we create an example python function. The function must take a single argument:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false }, "scrolled": true }, "outputs": [], "source": [ "def my_function(x):\n", " return x + 7\n", "my_function(3)" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "To start using `pywren`, we first create an executor." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "wrenexec = pywren.default_executor()" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "We can call `my_function(3)` remotely via `call_async`: " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "future = wrenexec.call_async(my_function, 3)" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "Future is a placeholder for the returned value from applying `my_function` to the number `3`. We can call `result` on it and get the result. Note that this will block until the remote job has completed" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false }, "scrolled": true }, "outputs": [], "source": [ "future.result()" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "You can apply `my_function` to a list of arguments, and each will be executed remotely at the same time. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "futures = wrenexec.map(my_function, range(10))" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The pywren `get_all_results` function will wait until all of the futures are done and return their results" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "pywren.get_all_results(futures)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's it, we are now familiar how to make use of PyWren for parallelly executing a Python function across many AWS Lambda functions." ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.13" }, "toc": { "nav_menu": { "height": "12px", "width": "252px" }, "navigate_menu": true, "number_sections": true, "sideBar": true, "threshold": 4, "toc_cell": false, "toc_section_display": "block", "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }