{ "cells": [ { "cell_type": "markdown", "id": "9adbbd88", "metadata": {}, "source": [ "# Fine tuning BERT for information retrieval using Amazon Sagemaker " ] }, { "cell_type": "markdown", "id": "b60614e2", "metadata": {}, "source": [ "## Runtime\n", "This notebook takes approximately 30 minutes to run.\n", "\n" ] }, { "cell_type": "markdown", "id": "9933f5b8", "metadata": {}, "source": [ "## Contents\n", " Background\n", "1. Development environment and permissions\n", " - Installation\n", " - Permissions\n", "2. Training\n", " - Downloading data \n", " - Preparing the data\n", " - Bi-Encoder Transformer Neural Network\n", "3. Inference\n", " - Offline scoring\n", " - Realtime endpoint\n", "4. OpenSearch\n", " - OpenSearch Client\n", " - Index and mapping\n", " - Ingestion of documents\n", "5. Simulated Semantic Search Application\n", " - Search Widget\n", " - Pipeline\n", " \n", "### Terminology: sentence, document, passage : All of these terms mean the same, the response for a query " ] }, { "attachments": { "huggingfact-SBERT.jpeg": { "image/jpeg": "/9j/4AAQSkZJRgABAQAASABIAAD/4QBMRXhpZgAATU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAAB//8AAKACAAQAAAABAAAC/KADAAQAAAABAAABVQAAAAD/7QA4UGhvdG9zaG9wIDMuMAA4QklNBAQAAAAAAAA4QklNBCUAAAAAABDUHYzZjwCyBOmACZjs+EJ+/+INXElDQ19QUk9GSUxFAAEBAAANTGFwcGwCEAAAbW50clJHQiBYWVogB+YACQATAAoADQAyYWNzcEFQUEwAAAAAQVBQTAAAAAAAAAAAAAAAAAAAAAAAAPbWAAEAAAAA0y1hcHBsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASZGVzYwAAAVwAAABiZHNjbQAAAcAAAAHuY3BydAAAA7AAAAAjd3RwdAAAA9QAAAAUclhZWgAAA+gAAAAUZ1hZWgAAA/wAAAAUYlhZWgAABBAAAAAUclRSQwAABCQAAAgMYWFyZwAADDAAAAAgdmNndAAADFAAAAAwbmRpbgAADIAAAAA+Y2hhZAAADMAAAAAsbW1vZAAADOwAAAAodmNncAAADRQAAAA4YlRSQwAABCQAAAgMZ1RSQwAABCQAAAgMYWFiZwAADDAAAAAgYWFnZwAADDAAAAAgZGVzYwAAAAAAAAAIRGlzcGxheQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAG1sdWMAAAAAAAAAJgAAAAxockhSAAAAFgAAAdhrb0tSAAAAFgAAAdhuYk5PAAAAFgAAAdhpZAAAAAAAFgAAAdhodUhVAAAAFgAAAdhjc0NaAAAAFgAAAdhkYURLAAAAFgAAAdhubE5MAAAAFgAAAdhmaUZJAAAAFgAAAdhpdElUAAAAFgAAAdhlc0VTAAAAFgAAAdhyb1JPAAAAFgAAAdhmckNBAAAAFgAAAdhhcgAAAAAAFgAAAdh1a1VBAAAAFgAAAdhoZUlMAAAAFgAAAdh6aFRXAAAAFgAAAdh2aVZOAAAAFgAAAdhza1NLAAAAFgAAAdh6aENOAAAAFgAAAdhydVJVAAAAFgAAAdhlbkdCAAAAFgAAAdhmckZSAAAAFgAAAdhtcwAAAAAAFgAAAdhoaUlOAAAAFgAAAdh0aFRIAAAAFgAAAdhjYUVTAAAAFgAAAdhlbkFVAAAAFgAAAdhlc1hMAAAAFgAAAdhkZURFAAAAFgAAAdhlblVTAAAAFgAAAdhwdEJSAAAAFgAAAdhwbFBMAAAAFgAAAdhlbEdSAAAAFgAAAdhzdlNFAAAAFgAAAdh0clRSAAAAFgAAAdhwdFBUAAAAFgAAAdhqYUpQAAAAFgAAAdgARABFAEwATAAgAFAAMgA0ADEANABIAAB0ZXh0AAAAAENvcHlyaWdodCBBcHBsZSBJbmMuLCAyMDIyAABYWVogAAAAAAAA89gAAQAAAAEWCFhZWiAAAAAAAABlpAAANDwAAAEtWFlaIAAAAAAAAGjMAAC7EQAAERxYWVogAAAAAAAAKGYAABCzAADA5GN1cnYAAAAAAAAEAAAAAAUACgAPABQAGQAeACMAKAAtADIANgA7AEAARQBKAE8AVABZAF4AYwBoAG0AcgB3AHwAgQCGAIsAkACVAJoAnwCjAKgArQCyALcAvADBAMYAywDQANUA2wDgAOUA6wDwAPYA+wEBAQcBDQETARkBHwElASsBMgE4AT4BRQFMAVIBWQFgAWcBbgF1AXwBgwGLAZIBmgGhAakBsQG5AcEByQHRAdkB4QHpAfIB+gIDAgwCFAIdAiYCLwI4AkECSwJUAl0CZwJxAnoChAKOApgCogKsArYCwQLLAtUC4ALrAvUDAAMLAxYDIQMtAzgDQwNPA1oDZgNyA34DigOWA6IDrgO6A8cD0wPgA+wD+QQGBBMEIAQtBDsESARVBGMEcQR+BIwEmgSoBLYExATTBOEE8AT+BQ0FHAUrBToFSQVYBWcFdwWGBZYFpgW1BcUF1QXlBfYGBgYWBicGNwZIBlkGagZ7BowGnQavBsAG0QbjBvUHBwcZBysHPQdPB2EHdAeGB5kHrAe/B9IH5Qf4CAsIHwgyCEYIWghuCIIIlgiqCL4I0gjnCPsJEAklCToJTwlkCXkJjwmkCboJzwnlCfsKEQonCj0KVApqCoEKmAquCsUK3ArzCwsLIgs5C1ELaQuAC5gLsAvIC+EL+QwSDCoMQwxcDHUMjgynDMAM2QzzDQ0NJg1ADVoNdA2ODakNww3eDfgOEw4uDkkOZA5/DpsOtg7SDu4PCQ8lD0EPXg96D5YPsw/PD+wQCRAmEEMQYRB+EJsQuRDXEPURExExEU8RbRGMEaoRyRHoEgcSJhJFEmQShBKjEsMS4xMDEyMTQxNjE4MTpBPFE+UUBhQnFEkUahSLFK0UzhTwFRIVNBVWFXgVmxW9FeAWAxYmFkkWbBaPFrIW1hb6Fx0XQRdlF4kXrhfSF/cYGxhAGGUYihivGNUY+hkgGUUZaxmRGbcZ3RoEGioaURp3Gp4axRrsGxQbOxtjG4obshvaHAIcKhxSHHscoxzMHPUdHh1HHXAdmR3DHeweFh5AHmoelB6+HukfEx8+H2kflB+/H+ogFSBBIGwgmCDEIPAhHCFIIXUhoSHOIfsiJyJVIoIiryLdIwojOCNmI5QjwiPwJB8kTSR8JKsk2iUJJTglaCWXJccl9yYnJlcmhya3JugnGCdJJ3onqyfcKA0oPyhxKKIo1CkGKTgpaymdKdAqAio1KmgqmyrPKwIrNitpK50r0SwFLDksbiyiLNctDC1BLXYtqy3hLhYuTC6CLrcu7i8kL1ovkS/HL/4wNTBsMKQw2zESMUoxgjG6MfIyKjJjMpsy1DMNM0YzfzO4M/E0KzRlNJ402DUTNU01hzXCNf02NzZyNq426TckN2A3nDfXOBQ4UDiMOMg5BTlCOX85vDn5OjY6dDqyOu87LTtrO6o76DwnPGU8pDzjPSI9YT2hPeA+ID5gPqA+4D8hP2E/oj/iQCNAZECmQOdBKUFqQaxB7kIwQnJCtUL3QzpDfUPARANER0SKRM5FEkVVRZpF3kYiRmdGq0bwRzVHe0fASAVIS0iRSNdJHUljSalJ8Eo3Sn1KxEsMS1NLmkviTCpMcky6TQJNSk2TTdxOJU5uTrdPAE9JT5NP3VAnUHFQu1EGUVBRm1HmUjFSfFLHUxNTX1OqU/ZUQlSPVNtVKFV1VcJWD1ZcVqlW91dEV5JX4FgvWH1Yy1kaWWlZuFoHWlZaplr1W0VblVvlXDVchlzWXSddeF3JXhpebF69Xw9fYV+zYAVgV2CqYPxhT2GiYfViSWKcYvBjQ2OXY+tkQGSUZOllPWWSZedmPWaSZuhnPWeTZ+loP2iWaOxpQ2maafFqSGqfavdrT2una/9sV2yvbQhtYG25bhJua27Ebx5veG/RcCtwhnDgcTpxlXHwcktypnMBc11zuHQUdHB0zHUodYV14XY+dpt2+HdWd7N4EXhueMx5KnmJeed6RnqlewR7Y3vCfCF8gXzhfUF9oX4BfmJ+wn8jf4R/5YBHgKiBCoFrgc2CMIKSgvSDV4O6hB2EgITjhUeFq4YOhnKG14c7h5+IBIhpiM6JM4mZif6KZIrKizCLlov8jGOMyo0xjZiN/45mjs6PNo+ekAaQbpDWkT+RqJIRknqS45NNk7aUIJSKlPSVX5XJljSWn5cKl3WX4JhMmLiZJJmQmfyaaJrVm0Kbr5wcnImc951kndKeQJ6unx2fi5/6oGmg2KFHobaiJqKWowajdqPmpFakx6U4pammGqaLpv2nbqfgqFKoxKk3qamqHKqPqwKrdavprFys0K1ErbiuLa6hrxavi7AAsHWw6rFgsdayS7LCszizrrQltJy1E7WKtgG2ebbwt2i34LhZuNG5SrnCuju6tbsuu6e8IbybvRW9j74KvoS+/796v/XAcMDswWfB48JfwtvDWMPUxFHEzsVLxcjGRsbDx0HHv8g9yLzJOsm5yjjKt8s2y7bMNcy1zTXNtc42zrbPN8+40DnQutE80b7SP9LB00TTxtRJ1MvVTtXR1lXW2Ndc1+DYZNjo2WzZ8dp22vvbgNwF3IrdEN2W3hzeot8p36/gNuC94UThzOJT4tvjY+Pr5HPk/OWE5g3mlucf56noMui86Ubp0Opb6uXrcOv77IbtEe2c7ijutO9A78zwWPDl8XLx//KM8xnzp/Q09ML1UPXe9m32+/eK+Bn4qPk4+cf6V/rn+3f8B/yY/Sn9uv5L/tz/bf//cGFyYQAAAAAAAwAAAAJmZgAA8qcAAA1ZAAAT0AAAClt2Y2d0AAAAAAAAAAEAAQAAAAAAAAABAAAAAQAAAAAAAAABAAAAAQAAAAAAAAABAABuZGluAAAAAAAAADYAAKaAAABWgAAAUgAAAJ0AAAAngAAAD4AAAFBAAABUQAACMzMAAjMzAAIzMwAAAAAAAAAAc2YzMgAAAAAAAQu3AAAFlv//81cAAAcpAAD91///+7f///2mAAAD2gAAwPZtbW9kAAAAAAAAEKwAAKCbNU1WVdJN0Y8AAAAAAAAAAAAAAAAAAAAAdmNncAAAAAAAAwAAAAJmZgADAAAAAmZmAAMAAAACZmYAAAACMzM0AAAAAAIzMzQAAAAAAjMzNAD/wAARCAFVAvwDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9sAQwABAQEBAQECAQECAwICAgMFAwMDAwUGBQUFBQUGBwYGBgYGBgcHBwcHBwcHCAgICAgICgoKCgoLCwsLCwsLCwsL/9sAQwECAgIDAwMFAwMFDAgGCAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwM/90ABAAw/9oADAMBAAIRAxEAPwD+/iiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP/Q/v4ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD/0f7+KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiivgf/AIKXfGX4lfs//sTeNvi58ItSOkeI9INj9ku/Kim2edf28Mn7udHjO6N3HKHGcjnBrqweFlia9OhF2cml95x5ljaeCwtXFVdoJv7lc++KK/hG/wCHyv8AwUk/6KT/AOUzS/8A5Eo/4fK/8FJP+ik/+UzS/wD5Er77/iGeY/8APyP3v/I/Jf8AiNuS/wDPqf3L/M/u5or+Eb/h8r/wUk/6KT/5TNL/APkSj/h8r/wUk/6KT/5TNL/+RKP+IZ5j/wA/I/e/8g/4jbkv/Pqf3L/M/u5or+Eb/h8r/wAFJP8AopP/AJTNL/8AkSj/AIfK/wDBST/opP8A5TNL/wDkSj/iGeY/8/I/e/8AIP8AiNuS/wDPqf3L/M/u5or+Eb/h8r/wUk/6KT/5TNL/APkSj/h8r/wUk/6KT/5TNL/+RKP+IZ5j/wA/I/e/8g/4jbkv/Pqf3L/M/u5or+Eb/h8r/wAFJP8AopP/AJTNL/8AkSj/AIfK/wDBST/opP8A5TNL/wDkSj/iGeY/8/I/e/8AIP8AiNuS/wDPqf3L/M/u5or+Eb/h8r/wUk/6KT/5TNL/APkSj/h8r/wUk/6KT/5TNL/+RKP+IZ5j/wA/I/e/8g/4jbkv/Pqf3L/M/u5or+Eb/h8r/wAFJP8AopP/AJTNL/8AkSj/AIfK/wDBST/opP8A5TNL/wDkSj/iGeY/8/I/e/8AIP8AiNuS/wDPqf3L/M/u5or+Eb/h8r/wUk/6KT/5TNL/APkSj/h8r/wUk/6KT/5TNL/+RKP+IZ5j/wA/I/e/8g/4jbkv/Pqf3L/M/u5or+Eb/h8r/wAFJP8AopP/AJTNL/8AkSj/AIfK/wDBST/opP8A5TNL/wDkSn/xDPM/54/e/wDIP+I25L/z6n9y/wAz+7miv4SP+Hyv/BSX/opH/lL0v/5Epn/D5X/gpQ8myP4isFH3s6Xpmf8A0kol4Z5gl/Fj97/yLh40ZPJ2VKf3L/M/u6or+Ehv+Cyv/BSHdub4isF7r/Zel5/9JOlRyf8ABZX/AIKR7iY/iRx/2DdKP/tpxUQ8NcwntUg/m/8AIuXjRlNN2nRnf0X+Z/d3RX8Iv/D5b/gpJt/5KR/5TNL/APkSl/4fK/8ABST/AKKT/wCUzS//AJErT/iGWY7c8fvf+RlLxuyX/n1P7l/mf3c0V/CN/wAPlf8AgpJ/0Un/AMpml/8AyJR/w+V/4KSf9FJ/8pml/wDyJU/8QzzH/n5H73/kL/iNuS/8+p/cv8z+7miv4Rv+Hyv/AAUk/wCik/8AlM0v/wCRKP8Ah8r/AMFJP+ik/wDlM0v/AORKP+IZ5j/z8j97/wAg/wCI25L/AM+p/cv8z+7miv4Rv+Hyv/BST/opP/lM0v8A+RKP+Hyv/BST/opP/lM0v/5Eo/4hnmP/AD8j97/yD/iNuS/8+p/cv8z+7miv4Rv+Hyv/AAUk/wCik/8AlM0v/wCRKP8Ah8r/AMFJP+ik/wDlM0v/AORKP+IZ5j/z8j97/wAg/wCI25L/AM+p/cv8z+7miv4Rv+Hyv/BST/opP/lM0v8A+RKP+Hyv/BST/opP/lM0v/5Eo/4hnmP/AD8j97/yD/iNuS/8+p/cv8z+7miv4Rv+Hyv/AAUk/wCik/8AlM0v/wCRKP8Ah8r/AMFJP+ik/wDlM0v/AORKP+IZ5j/z8j97/wAg/wCI25L/AM+p/cv8z+7miv4Rv+Hyv/BST/opP/lM0v8A+RKP+Hyv/BST/opP/lM0v/5Eo/4hnmP/AD8j97/yD/iNuS/8+p/cv8z+7miv4Rv+Hyv/AAUk/wCik/8AlM0v/wCRKP8Ah8r/AMFJP+ik/wDlM0v/AORKP+IZ5j/z8j97/wAg/wCI25L/AM+p/cv8z+7miv4Rv+Hyv/BST/opP/lM0v8A+RKP+Hyv/BST/opP/lM0v/5Eo/4hnmP/AD8j97/yD/iNuS/8+p/cv8z+7miv4Rv+Hyv/AAUk/wCik/8AlM0v/wCRKP8Ah8r/AMFJP+ik/wDlM0v/AORKP+IZ5j/z8j97/wAg/wCI25L/AM+p/cv8z+7miv4Rv+Hyv/BST/opP/lM0v8A+RK/s3/Zb8ZeJfiP+zN8OfiF4xuvtusa94Y03Ub+42onmz3FpHJK+yMKi7nYnaigDoABXg57wnisphGpXad+1/8AJH13CfH+Bz+rOlhIyXLrrb9Gz6Cooor5U+4CiiigAooooAKKKKACiiigAooooAKKKKACiiigD//S/v4ooooAKKKKACiiigAooooAKKKKACiiigAooooAKK8M+J3x++G3wrjkh16+WW9UfLaQfNJnsGAzsz6tivz6+IP7a/xH8RyS2ng1E0W0OVVlAkmI9ycgH3Wvz/ibxKybJW6dapzVP5Y6v/JH1uScG5jmXvUo2h3eiP1Z1XxFoWhQtNrN7BaqgyfNcL/M14H4k/a4+B/hxjF/abX7r95bVC36nC/rX47ahqHi3xpfNdapcXGpzn+KUvIR+ecCtix+HPiC6/1wWFf7zEZ/IfNX4lm3j7jpu2CoKK89X+h+iYXwywVFXxle/pofopq/7e/geFtug6PeTr6zlI/5F65Gb/goJtH+i+GN/wDvXGP/AGma+O4PhTJ/y9XQ/wCAA/1rSX4V6Z/y0upf+Agf1r4mv4y8STbaxFvRL/I9eHCfDtPR07/Nn1vb/t/xP/x8+GmX/dnz/wCyCu40X9vH4ZzqF1rT7+2J/uBJAPqS6n/x2vgyX4V6f/yzun/4EB/Ray7v4V3X/LrdIf8AZYHNaYbxn4ipu7r39Uv0SFU4Q4eqqyjb5s/XHwx+018GPFjrBY61HbzP0iuAUP5/d/WvbrPULPUIPP0+ZJ09YyCP0zX8/V94D8RWe79x5w/vIQf0HNTeGfHnj74e3iyeGNQuNOkRt2xSQv4oeD+Ir7rJvH+unbMKCa7x0/B3/Q8TF+GGGqpvA19fP/gH9CFFfl18N/26PEFpIlj8TLJbuHgNdW4CyD3KDgj/AHRX6BeBPid4G+JOn/2j4O1GK7VR86A4kT/eQ8j8a/dOGuPMozuywdX3/wCV6P8A4PyPzbOeFsfluuJp6d1seh0UUV9ifOBRRRQAUUUUAFfmB/wWV/5RufEn/uGf+nS0r9P6/MD/AILK/wDKNz4k/wDcM/8ATpaV7PD/APyM8P8A4l+Z8/xd/wAiXGf4Jfkz+Eaiiiv6nP4LCiiigAooooAKKKKACiiigAoo2/w4o27eOlHkVysKKKKCQooooAKcu7Py02jdt56Ua9Cole8uIrG1mvLpv3MKGV/oBk/yr8r9Pvv2qv8AgoP8cpvgv+zVC/2S3Z2ba3lRLEpI82eTBwD298Cv0y8daTceIfB2p6NZ8S3FvJGrZxzjIGf/AB2vlv8A4Iw/tgfDH9jn44eLfAvxo26VNr52W93OMBXU42OfmwCBmvguLq9V1oUublh3P27wxwdB0K2L5OarHZHgP7QX7LP7eH/BNO6sfFXxAuUutMmcK1xZzm6t8k8JKSOCeijdX318G/iVpvxd+Hun+ONLGwXS7XTOfLkA5BPqK9e/4LI/t3fAfxp+zvffBvwrrFv4i8QeILqGdUtyJBBHEclyRwCc4VRzxzivi79hrwrq/hH9n+yt9YRopby4kuEV+CFkxj/0Gs8hbw+Y+wo1OaNvU7+OKMcTkyxuIpezq39Ln2E25Sw7Ckpzfe2r02/eptfoPMuh+Bz3CiiiggKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK/0Vv2Iv8AkzD4Q/8AYlaN/wCkENf51Nf6K37EX/JmHwh/7ErRv/SCGvy7xR/3aj6n714F/wC+Yj/CvzPqKiiivxA/poKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP/9P+/iiiigAooooAKKKKACiiigAooooAKKK4bx14+8NfDfw7N4n8U3Cw20Q4Hdj2VR3JrHE4mlh6TrVnaC3ZvRozqzVOCu2b2ua9o/hnS5Na16dLW2hG53c4AFfmJ8bf2zNa12ebw/8ACstY2PKNfNxLJ7oP4FP8J5JHpXgvxu+PPij4z60WuS1tpUbf6PZqeB6F8dXPvnHbFc74X+HslzjUNcXZF95YuhP1/u1/JfiB4x18c5YTK3yUu/WX+SP3DhzgbD5dTWKzLWp26L/NnF6bouueKLwzRBpXdtzyuTjJ7kmvWNH+G+l2aifVG+0yf3eiD8Oc/wDjtegRW8NrCtvbosSD7qqMCpq/n+dZu59Xic1q1NIaIht7e3tYxHaosYX7qqMVNRRWJ5TlcKKKKBBRRRQAVRvNM0/Uo/JvoUlHuKvUUFRbTunZnk+tfDGKTM2iy7D/AHH6fgf/ALGuB03UvFngHWotS0mefT7yBtyuhwf8CK+lqz9S0nT9Ytzb6hEsq/w7uo+h6iuqjiqlOanB6nq0M1dvZ1tUfS3wM/bJsPEE0Phf4p7LO7bCxXoGI5D0w4/gP+10PtX3zDMlxCs0LK6HlSpyCPrX8/HijwVeaGxurfdNaf3u4+v+NfSv7OX7Uuo/Dq4i8I+OZnu9EdgsUrZLwZ/Up7c47cV/S3h34ySg44LOXddJ/wCf+Z8NxRwFSrweMyn/AMB/y/yP17orL07U7DWtPh1PS5VuLe4UOkiHIIPcEVqV/T8JqcVOB+LuLTaas0FFFFMkK/MD/gsr/wAo3PiT/wBwz/06Wlfp/X5gf8Flf+UbnxJ/7hn/AKdLSvZ4f/5GeH/xL8z5/i7/AJEuM/wS/Jn8I1FFFf1OfwWFFFFABRRRQAUUUUAFFFC/L0oAKKczbq4H4mfEjwr8KfB9z4w8VXHkQQ/KqqOZCewHrx1rKviY0YurVdkehg8LVxVVUaMLtneUV+SWsf8ABR7xJeX0t94T8NPLpkJC+a2TwO5I6FsZr7L/AGef2qPBnx+tHt7NfsmqW6+Y9sx5PqU9fcV4GB4twOJq+xg9T7PNvDrOMvwyxVel7h9RUU6Q7m8tRyfl3dOtNr6aV/kfATjoFFFFESYkjKrMNzMFOPmr4I/bD/Zn+G/irwfq/wASo4/seq2UW9nhBAlOcEsDxn6cnvX3r97qM182/tT+NPDPhz4N6zpuqXiW9xdwYiicjcTnsK+f4jo4aWEftz7TgjF4ujmFL6vc/Or9hP8AZt8CfE7T7rxx4vEkzabeeQtuCPLbABBI69/4eK/Z+3tbezhS1tUWKKJQqooAAA7AD5QP9mvy5/4Jt+MfCdl4T1fw5eXiR6hNfmRIGIDMpAHGevT0r9TV+795T/P6H3ryeCaGHWDVSm9T6TxUxWLnmcqVZ+6thKKdt+X733qbX2h+TyCiiigkKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK/0Vv2Iv+TMPhD/2JWjf+kENf51Nf6K37EX/ACZh8If+xK0b/wBIIa/LvFH/AHaj6n714F/75iP8K/M+oqKKK/ED+mgooooAKKKKACiiigAooooAKKKKACiiigAooooA/9T+/iiiigAooooAKKKKACiiigAooooA5jxP4l0jwfoV14k16YQ2tqhkkc+noPU+lfib8cvjVrnxn8Vfb5d1vp1uxWzts9B6n1c9T6dPevYv2uvjnL438Qv4A8Ozf8SnTH2ysh4mmHU5HVR91exzn0rwz4f+E1bGuakmV/5YKfbv/hX8f+L3iI8yrvLcHP8Acx3/ALz/AMkfvfBXDlPLcN/aOLX7x7eS/wA2angvwTHYqmqasu6Y/MiN0Uep9/5V6dRRX4DKVz2q+JnXlzzYUUUVJzBRRRQAUUUUAFFFFABRRRQAUUUUANkVZFMcgUq3ysp6V4b428F/2XnVNLUtbt99F52f/W/lXulNaOORTHIFKsu1lboQaqE7HXg8ZPDzui/+y1+0bcfDzU4vBHi6Yvot04WORjnyHPf/AHD/ABenXmv1zR45Y1kjOVPTvmv59/G3hltBvvtNqM20zZX2Pof6V+in7G3xzm8Tad/wq/xRNvvbBM2bv1eJeCme5T+H1GfSv6f8GfEJqSyTGy0fwP8AT/I+R494Yp1aTzbBr/F/n/mfflFFFf0+fiwV+YH/AAWV/wCUbnxJ/wC4Z/6dLSv0/r8wP+Cyv/KNz4k/9wz/ANOlpXs8P/8AIzw/+JfmfP8AF3/Ilxn+CX5M/hGooor+pz+CwooooAcvLZprYVttKFLUlVEpWCn/ACf3eaVlCqefl4+ao8/dy3uuaNt1oVGF2G3dx1o+b6f7VORmX7ytuP61RTUdN+2fYJbiIT/eaJmG7HrjOcNUOvBPU3hhqkn7iLn8Jk+Vf0r8fP8Ago1460/WvE3h3wHp96ktqjFrpUOcFiACccAgZxX07+2Bq3x8kutO8JfCeKVrO/TdLPB1DEkEEg8DG2vmvT/+CdPiLxF4Pu9Y8YarjxBMu6CJSSNx5+c85/8Ar1+ecW4nGYynPCUIn7X4e4LAZZVpZrjaq9D9+vg78HPgD4F/Zz8P+GfB+lWF7Ff6bCzzqoZpTJGDIXPv8x2/hX853xEtPB/7Pv8AwUBfS/ANwlnpKahCs6g/u41lRDImR25P0q1oN/8A8FBPg3ob/Czwqbw6fHlEZMMIwT0Qk8D+76VueB/+CffjTxxpd/4v+KWpvb63e/PArHcRJ1Jc7u/THNfivD/C2bYbHOrZn9g+I3ilwtmORUsHSUVpa5+wen31jq1ml9p8q3EZxsdCCCD7jpirO5SOm07vm3cYFfiYupftNfsf659nuo31HR1YMysC8TAep5xxX6BfBn9r74Y/Fy3/ALNmuP7P1A4VoJiACfY9P/Hq/oTLuJac2qNdWmfwrn/AlelH6zg37Wm+qPrD5fm/2adyo+tRxtHIoMJ3LtG3byD9D3/CpM/3uG/utwR9c19XGpTesJH53Oi07WMXxFq0eg6Fea1cKpS1gZ8N3wM/zr88P2Df2M/GP/BVz9ozXrvxtrEmm+GtDbdcPEMuATtSNBkDgYLc9M19YftBfFbwD8OPAuox+MLlfNvLdkjgjIMhJGBx6Z/ir89P+CZn/BRzxB/wT6+K+p+JrnTJb/wx4i+S8iUEEHPEiH+8AOxr8F8YsdjZYXky+Xv+R/UngPlOFp1HisdHR9T7y/4Kd/8ABG/Sf2HPhf8A8NDfs++ILq7s9JuI4L2C5GJIy54kQgnIzuJ9BUn7J/xUvvi98F9P8Uaoqm8iY2s7L3ZMZP45rH/4KVf8FjLH9tv4Ww/s4/APQri2stWuI5b64nGZZPLPyIPvEDJbce+ea2P2VvhbefCP4N6d4X1JcXchNxOvo0gGR+GK+a8Bv7b9n/wot/M+i+kP/Y3sl9TtdbH0cy7fu/dam0UV/UMtz+NJ/ZCiiioMwooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAr/RW/Yi/wCTMPhD/wBiVo3/AKQQ1/nU1/orfsRf8mYfCH/sStG/9IIa/LvFH/dqPqfvXgX/AL5iP8K/M+oqKKK/ED+mgooooAKKKKACiiigAooooAKKKKACiiigAooooA//1f7+KKKKACiiigAooooAKKKKACvmX9qL4tP8Lvh1KNMbZqWp5trb1GR87D3APHvX01X4l/tVfEib4gfFi6ht5d9lpObOBR0yp+cn335GfTFfmfirxS8myaXsX+8q+6v1f3H2vA2R/wBo5gvafBHVnifhXQ5vEWsCGTcYx88re2f6mvpeOOOOMRxjChdqqvYCuL8C6L/ZOhLJIMS3Hzt6gdh+FdrX8IVZXZ+1Zniva1WlsgooorI8wKr3F1Z2KiS+mS3U/dZ2Cg/TJFfB/wDwUK/bY0n9iv4Op4ks4UvfEutObfSLOQ4UsB88r/7EeVyvUkjtmv45/ix+1V+0R8bvEE3iT4keL9SvZZmLLAszpAmTnEcSEIg/2Qor9f4B8Isw4kw7xntFSpd3rf0R8znXFFDAy9la8z/QMhkhuIxNbusqN91kII/McVJX8Hf7Ov7fH7TX7NPiiDWvCfiS81HT0YfaNJv5Xmt5I88gI5IQkdHXn3r+039mf9oDwf8AtP8AwX0X4zeC9yW2qIfNgb70M0Z2SRn/AHHBAb+IDI4NcHH/AIX4/hjlq1ZKdKWnMu/Zo3yTiKjmF4xVproe70UUV+XnvhRRRQAVTk1LTY5vs811Akn3djSID+ROa/nj/wCCqX/BUPxt8O/Gl5+zV+znff2dfWKhNZ1eLBkjkcZ8iInoQCCXGCMgA8EV/O3ffF74tarq39vap4p1a4vi25rh7qUyZ9d5Oc/jX71wj4FZjm+Bjja9VUlLVK13bv0sfG5nxhQwtZ0aceax/ombdvHSiv5M/wDgnX/wVc+KHw58faZ8J/2i9Xn17wnqkot0v71jJcWcjHAcynLvH90MGJxxgcmv6zK/OuNeB8fw1i1hcZqns1s/66nvZTm9HMKTqUtGFFFFfFHqGfq2m2+sadLp90vyuvyt6Hsa+e9H1XXPh74ug1rTWaG90+cOpU9cHkfQjIPsa+lK8j+J2iFli1yEfMPkfb+hP/oNdWFxE6VRTg9T2cqrq7oVFoz9pvhx41sPiH4J07xhprfJexBmUdmHDD8GBrvq/Mb9hX4jSW+pX/wuvnykwN1aj0YDEg+mAD9c1+nNf6DcC8SrO8ppYxv39peq3/zP584oyb+zcfUw3Tp6BX5gf8Flf+UbnxJ/7hn/AKdLSv0/r8wP+Cyv/KNz4k/9wz/06WlfpPD/APyM8P8A4l+Z+e8Xf8iXGf4Jfkz+Eaiiiv6nP4LCiiigAp3f60J96nK3zLntR3Za2bI5GSP/AFrYVF3NX5kfHr9s7xtH43/4Vf8AAeyOoXqNtaWBTM/mE4KAAEcfL/DX6OeKPtT+GdR+x7jJ9kmVNvXODjH49K/Pj/gkrq3w48M/tIeIV+JLImsSpJ9i+0Y5bJyBnv8Adr8w8SuIsTl2GXsUf0Z4B8B5fxDmXssa9D5o8cftOftl/CqKTSfiZp0+mG7T909xDgD6HAH/AAGvZ/2P/hH8SPGXiCx+PHjDWXlt5GLIiyZL8EYOWIA/2a/Rr/gql4o+Hus/s86hJ4oaB7x3RdOVSCwb5uRjnpuzXyD/AME7Y9Vh+BMUl+P3P2iTyFbOcEnke1fEeH2cYnN8SvrTdj9j8d+DMBwbQlDLGve8l1PvzduH8Kry21ccEfzyPSmtjCxhlC/e3KD/ACNR0V/QvJHex/Cftp2tfQdv2sZAWb5ttObazbsLhfmXb1z2NR0Uckf5QdeVrFPVtJ0vXbOSw1iBLuGZdrJIAVP1zyP+A4r87/jV+wboOrfaPE3wjuDpV7DktDk7Tjng9RX6OVi+KGuJPDt+9nuE3kSbPrg15GbZXRxNJuoj6vhfiPF4CvFUJWTPwt8IftQftK+C9Qk+Evhsy6xfRS+QiIvnSAg4wCATXputftTftifBmTb8V9Gngju1Ji+2QlQSR6gckf3d1ezf8EoNc+G/h39qfxF/wswpFq0jTfYmnwRnLZAz0J7V+if/AAVF8U/D3Vf2fdWm8UNbrd7glknHmFxnlO/pmv5lxnF+Y4bG/VactD/Q3IfBzJMzyCpnFa3tF0sj8d/gr8DfiB+1hrn/AAs74nXzvpO/PzHluegHQD/gNfqhJ8A/hLJ4bg8K3GhW8tpbKNu5eRnjOepr5d/4JzxaxD8E5F1IOYTcboV7456e1f0bf8Eov2Y/hv8AtTftMT+CfitaNfaZpth9se1YYWQk7AH55A6/LX7hldDCYfL/AK5iVfS7P4v4mzLH4vOv7MwcuVJ2Vj8bfBPwH+E/gW8XUPDuj28FyW+V8Akj2znH4V7Ay/MD83z/AHV6/gK/oG8afE7/AII1+BvGWp+B9U+FzfaNKupLOVkBwGjJBxz0zXD/APBSz9lH9l/wb+zF4L/aS/Z10FfD8HiF41aCIn95HMhdCQT1GD09a9HBZ5SpVYUlRcOY8DNeHcViYSrPEc/KfhbuXco3fMfu05VLfd5r+gP/AIJ8/s4/sh65+xP4h/aI/aI8Jwa5JolxIzu+fMESkoAADxxtq/8ADLx5/wAEcfit4+0f4e6H8L2S61u4jtYGkB2gyEAZ+bpk1pW4pSqzjGk2onHQ4Gc6VOrOslzH89vP503Z/n9a/ST/AIKmfs4/Df8AZj/aY/4Q34UWv2LRbyyivI7VTkRNISCB/FggLjmv1U8SfAP/AIJx/s5fsl+AvjJ8c/h7Bqk/iGzh82VMktM0e8k898NW9biSlChSrqN+c5sHwZWq161GckuQ/mJ+Yt8393d+FNXae/8ADu/A96/XP9o74tf8ExfEXwn1HR/gH4CbSPE0yhbS4AICnjqSW7f7Nez/APBLP9mn9mH4pfs++OPip+0N4ag8QDw7MWV5STII1TJAA2/hVVuIPZYX6zVptGWH4TdbFrC06qZ+Ev8ACZP4V+Vm9KPTbzu+771+92l/Gj/gjHq+tWOk2/wtcTXc8dujMDnMhCDPPqa8v/4Kdfsx/s9/sufHD4f6x8P9HWy8Na6sV5f2SE7TCJF8wAHkb0Bzz1NZUOJlOsqNSDVzoxHBbp0JVoVU7H4w/KQB/eob5Qd3y4XcfpX9M37K+k/8Ek/2qPipafCHwT8Mki1KezkuFeUEKRCAXzz154rx39pLxJ/wSa+D/izxd8H7X4bLF4i0RprJZ4gcCYDAIJzwDg9KxhxS54h0PYu5tW4HjTwv1p11Y/n92t0ptNjjkjXy5dzsMfN1BPc54x6U6vrlK+p8DOnZ2vcKKc23+Gm1Wm/2RcgUUVInH90N/eakLlRGp+batG3+LFOUNIp2hiq/dXqc/hSem5cHvTjqOVNpXEooopcr6kyCiiigkK/0Vv2Iv+TMPhD/ANiVo3/pBDX+dTX+it+xF/yZh8If+xK0b/0ghr8u8Uf92o+p+9eBf++Yj/CvzPqKiiivxA/poKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP/1v7+KKKKACiiigAooooAKKKKAPLfjD4wHgH4Z6z4r/itoCEA4O5yEXH4nNfhXoNnceIfEMcd0WlMzmSVm6nnLn8ea/T79u3xR/Z3w7sPDELbX1G6Dt7pGDkfmVb8K/PX4W2fmXlxqDD5Y1Cq3uev6V/HvjtnLrZvHCJ6U4/i9fysfvfh5hVhMpqYt7yf5afnc9qK9P7o+Wiis3XJprfR7qa24lSJmVvQgda/AbeR6kIuTS7nP6r460HSLhrWQtNKPvLEMgexNa2i+INL16PzLB8sPvK3BH1FfLrMWZmP3j8xrqPBNxcW/iS2FuW+dtrL6g9c10yopLfU+krZRTVK63P55f8Agvdda5J+0B4Rsrzd/ZsekFrfOceYX/e47Z4TNfg/X9sH/BTv9hu4/bI+EdtL4LZI/F/hppJ9NZyAJo5APMgJPA3kKVJ6Edea/nv/AGJ/+CZPxi+N3x3/AOEZ+NHh7UfDnhrQJd+ry3UckJl2niCJyAH3sNpdM4ByDX9oeF3HmU4fhWPt6qi6CfMuu+ll1v8Amfz7xFkuKnmP7tXUtj8rgyt91t1f1Y/8ECrrWpP2e/G1vfFjZw+IEW13dADbgvj2znPvX2Z8Sv8AglX+xH8SPD8Wht4Qi0WW2QJFdaaxhkBAwC+whZD67+tfTH7NP7N/w6/ZV+Flp8Jfhmjiytnkneec5llkkJJdz/FycAdAOBxX534j+L2V8QZG8Fh6clUclvbZdb3PcyLhfE4LGKtUase+UUUV/NR9+FSRbfPTPTcP51HRQB/ng/HK9168+NXi+68TMxvn1e781nzniUgZz/sBa8t3K33Tmv7ZfHX/AASf/ZH+JXxw1T43eNNPvLu41WUXE1gkzx25mAAL4Qg4IxuHQnnHNfPn7fX/AASj+EvxA+Dc3iD9mvw9b6D4s0JDPBb2YEcd3Go+eJ0GAXxyh6kgDPNf2zk3jvkdSeHwbjKN0k3pyxf37eZ+S4vg3GJVKqadvvZ/JDGzedF5f3lcMv1yMV/oW/s83WtXvwF8F3niPd/aEmiWjXG/r5hiXfnPOc1/J3+wH/wTF+L/AMdvilZ+Ivi/od54d8H6JdCW9+3xvDLcGM58qJHAcgt95wMD1r+yGGGO3jS3t0VIkUKoUYAA4AA7Cvz36QfE2BxlbD4HCyUpwu210vbQ9zgjL6tKE61RWTMHXPE2k6Cq/bnbe3zKijJIqjovjTQ9auPssLtFKfuhxjP0rw3xPcXF1r1zJdFmIfb83YDpWHHI8Mgmhba4bcrL6jpX84xoq2+p+y08mpukrvU+vKz9YsF1bS7jT2XPmIVX69R+tTWMsj2MM0n3mQM31IFWq5kfN3cGfPvw48WXfw7+Imm+KIywNhcBpVXuoOHH0I4r+gCCeO5gS4hIZXUEN2Oa/nu8bWP9n+JLiNRhXYOv4/8A181+1f7Ovif/AIS34OaHqkjM0sduIZWbu0fBr+n/AAAzhqriMBJ6NKS/J/ofNeKOEVSjQxsV5fqv1PcK/MD/AILK/wDKNz4k/wDcM/8ATpaV+n9fmB/wWV/5RufEn/uGf+nS0r+uuH/+Rnh/8S/M/nXi7/kS4z/BL8mfwjUUUV/U5/BYUU/5dvvTNu7jrTV72iioxHJ96m05PvU2jllccYkjrtX5hkFdu2vyv/aw/Y9b7RffGf4a3/8AZ9zap588GcAkZOUI5z/wEV+qDSfwqflb7y9q8g+PW1fg/r7R/d+xu23HcCvnuJsuo4vCSVZH3XAmf4vLMyhLDytc/Gj9n/4G+PP2qtYl1Txprks2n6awV/PYtIfYZ6Djn5q/c7wd4R0PwH4dtfCvhtFhtLNAiKOM46k+pNfnP/wTdG3Q9e8s/KXCsvrya/T5e3svHTpXi8D5TQw+F9rTWp9Z4t8T47G5g8PXqXSG0UUV90fiwUUUUAFSeTuQLjd1ZlbuDUdSKzKoXO7FPlv7r2NYSt7x+Xf7Vn7HazXF78ZvhveLptzAvnzwcgkjkkEfxfxV8l/AX4I+Pv2sNUlv/HGtyvp+juEfexkkPsgPGOP7wr9nvjkq/wDCpfEHy7s2knc9ge2a+DP+CaHzaT4nMi7m+0L/AFr8nzLhrBzzulBre/5H9I5Dx5mtPhXETjV+Gy/E/R/wN4L0XwD4ZtPCnhxEjtbNAqso5J9x6/3jX7jf8EKfFGh+Hf2xLqx1y4S3fUdJMFvvOMyBy5CHuSAxx6V+NMaLyw5LfhW94T8U+I/BXiSy8WeE7x7LUdNnE9vLAcFWU5BBHJHHKngjg8V99mGUrE4N4SmrXR+JZTn0sNmKxtXWzufQ37a3wt8bfCX9pjxlo/jmxawmu9SmvImbJEkcjmRHT+EghlHB6iv0C/bM/al+CfxJ/wCCefw0+DPgvW4r/X9Ea0+2WYBBjEcbg5JXGQSv3TX2VDL8Of8Agsl+y2LaQ2+l/GDwfAFVsgGTHTJ6FJMY9AcnjNfzV+MPB/iL4e+JtU8G+LLJ7LUNNlNtcQSr5ckbKcEEEA89a+cy2VLHNLEaVKXQ+uzaFXLYyqYR81Kr1P6DP2P0b/hzf8SjjLCWb7vp5mK/F/8AYxbH7U3w6G9Sx1m0/DMg4P8ADz7NX9AH/BMnQfhj4m/4Ji+MNF+MmoPpXhu4vJlvbqAEmNPMOCBtbPO3+9Xb/sPfsz/8Ev8ASfHUvij9njxLH4z8W6agl0631SQIFlAypjjIQFs/eKqdvXivAWb08LLE03F6v9D6x5DWxlPBVac0kl1/yPzg/wCC5km79r7TmJ5GjW7bhkbRl+vrmvpL/gpc27/gmp8GNv3dlv8ArbkV+dv/AAVS8TfHTxV+1dPN8eNBg8PX8dvHFZW9rIHU24J2PvBOSedy9vSv3k+Mn7PvwR/aA/YH+FGh/HDxyngaxtrO3nguHaJRI/lY2Ayqw6Etx6VvOqsNhcFKff16BhKM8XjMfCit49dOp/Hqien3R938K/oh/wCCV+7/AIYH+NEn8W2b/wBFnFfEX7YH7G/7LPwL+GEXi74K/FSLxrqr3Cp9iSW3Y+WSAXxGueB+FfbX/BK2RT+wP8aPlYsEmZlUg/8ALM/4V7XEePpYnAKpSWl1+Z8jwtllbC5jKnV3s9n5H4C+Cfn8aaIv8LapbHb2yJ0r92v+C7TN/b3ww9tGP9ad+zX/AMEy/wBkLxl+zboH7WHj74ka3otvHLHcXgVbcwxSxuH2EmNiBkYOTXhP/BYD9pX4M/Hz4geFNL+Durf2xZ+GbL7LPeKpEZJJwEJ4Jw25mXiuaOMpY3M6Kw6+G99D0MRllbLMprrFNLntbU5//gii279u/TPM5zpF7/6AlfK//BQRmH7bHxMXPyrrk/8AOvqb/gifuX9vDTP+wRff+gJXyv8A8FBP+T2viZt/6Dk/869DD3/t6p/h/U8jF/8AJOU/8R8h7ivT7v8AdqrcXUMLpHcSqm5c4bAOPerCbl+71HNfv9/wR98E/CPVvgf8TvGHxO8LWHiQaTB9oVbyGOVygQkojyAlCcdq+gzvNPqNH21rnynD+SPMq/sE7H8/8TR3m2O1DSNnb8g3YA74HI/FaXcyqV2rlDtPcAep/wB3+Kv6Vf2Xf+Cgn7NvxV+N2hfA/wAJ/A/Q9Ih1e6Nq1y9vEzgcgknkk8fxNXwT+3F8I/h/of8AwU4l+G+g6fFZ6HeXdnvs7cCKMeaQXAAwBk/3a8jC8RVZ13RxFO2lz6PG8HUo4WNfDVeZc1j8k5L23ClvOTK4+XI5+g61NHNDMXWGWKVVXcy5yR9e9f1IftrftOfAX9h3x9pPwp0P4KeF/EEbabFOtxdWcXmcjGN5HJ43bmavGf23tL+Dvxm/4J66J+0z4Z8D6T4S1a5uhtXS4Y4sAclCYwMg+7GsKPFNWbjelaEttToqcEU4qSVa846tWPGf+CTvwr+F/wATPhL8W9U8aaTZ6xPptkWt3uIwxj/dk5QnLDkD5vlr8UtQkg/ty6it2Tak7KqrjAAJwDX9Xf8AwSt/ag8H+PPgH4v0/T/Aek6S3hHTgtw9vEAb3EZJM5C/PnHO6vwS/bS/aq8H/tK+JNPuvCngPSPBf9lNJFL/AGTEkYlJc4LgYyef4q5coxuKnmVWE46ep1Z/leBpZPRnTlr6HxE23tTaXasf7v5dq/3e9JX3Z+UyCiiigkK/0Vv2Iv8AkzD4Q/8AYlaN/wCkENf51Nf6K37EX/JmHwh/7ErRv/SCGvy7xR/3aj6n714F/wC+Yj/CvzPqKiiivxA/poKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP/9f+/iiiigAooooAKKKKACiiigD8rP29dY+1+NNE0Nf+XO3d2X/roRj+VeDfDO38vQZZm/5aSlvyGK9E/beuN/xsaIf8srKH9c1xnw//AORXg/2mZv1NfwH4m4l1eIMZN/zW+7T9D+lMnpqlkGHiu356nZ01gG3Ky5U9VanUV+dnOeSax8M2mujNo8yohbcyP2+h9K6Pwv4Jt/D7fa7h/OuCu3djAAPp/wDFV3FFXKrNndPMK04+zb0ClZv71JRUHCFFQ2t1a30P2ixdJotxXehBGR1GRxkVNQAUUUUAFFFFABRRUP2u1+2GxMqfaAu5kyNwB7kfex71XKwJyxakooqQOC8T+BrfX5vt1m/k3H8WRwf8DWNoPw1WzuhdavIsio25UXoSPU16tRV+1nHS53QzCsqfs09AoooqDhPEfilb41K3utv302/iCT/Wv0f/AGGNYN38J59GJ3NZXjn8JORX54/FRR5dm5/vN/Kvtj/gn9MG8PeJLf8A553ELfmH/wAK/ZvBTEunxFRgtpJr8L/oYcaUlU4ebfRr87H6G1+YH/BZX/lG58Sf+4Z/6dLSv0/r8wP+Cyv/ACjc+JP/AHDP/TpaV/efD/8AyM8P/iX5n8tcXf8AIlxn+CX5M/hGooor+pz+CxVXc33ttPWPKn5dwX7xXoKaqfKdw+83yt6e34mvgH9rr9qDxD8PdUtvhd8MY1n8QXyhWZRuMe44CRoOS5/ve9cGa5pTwdH2lRn0nD+QV82rrC0D7++9yG4/ham/whuzfdr8z/Fn7I//AAVe+GHgFP2gPFVlq0emqgvHiaYSSLERkExg9P8AZzmvbP2T/wBplfjT4fk0vxC6W+tWK/vlUgLIAcZweQc7ePevByviyliavsnGzPr8/wDDnE4Ch7elNSS3t0Psjyt27leK8h+PXyfB/wAQ/wDXnJ/KvTptSsbWEyTTIiopZtxGOOfXsK/JL4zfG744ftIfEiT4Bfs+WctzBOCjrbgFpDnByc4CD/aNacT51h8FhX7V7nN4fcI43NsxjGitFqd3/wAE3WVtF15R3lG33yTX6gfdXd93DbW3V+Gl58P/ANrz/gn7q1vrHizS3s9MvWHmrxJGc9iR0P8AdOK/X74Q/EzRfix4FsPGml8JcrteJTnDAcgn2rxuBM/oYmh7Bbo+t8YeDsfgMb9brR0Z6XRRTtjV+hH4QNooooAKdsam024by1LXD4ULu3ewGTRzJbm1GDm+Vbs8t+OX/JJ9eb7qtayN830NfBv/AATT2/2L4n3Mq/6SrZ/OuN+NXx++M/7QHxKuPgD+zzay3KzMYHW3GZJSOHJOcAD39K8zuvhz+15/wT9votc8Waa9rpuoOPNDESQk+hx0JzX47mfF2FhnVFv7Nz+osg8NM0q8K14qNuaz/E/cddrRkt0/nj0r6x/Y9/ZH8XftnfEy6+GPgu9t9MubSz+2PLOCR5e/GAAQa/Pv4N/FLS/jB4DsfGWlqqNMoWVO6tjkDHvX9B3/AAQVmht/2utWW4ZVZ9EP3jjpJnH1r9DznNHDL5YrD9j8RyDI1PNo4LFq1mfmf8Pfih8RP2Mf2iD4m8D3ijU/Dd/JazqhIinEb7HjkGeUOPqM8Gv1+/4K6eAfAXxM+A/gD9uLQbBdL1bxKkMF7EmP3olQuhc4wSmGG7qc+1fGvxy/4Jz/ALZniT41eKPEWg+Cb+a2vdWuJ4J0MfMbSEggFh2Nfol/wU68O654D/4Jk/CbwT4wgNnqtlLaQXUDkbkkWJwQcHsetfMY3GUKuPw9XDPV7/cfWZXga0MuxNDELRao5v8AZB8xf+CNfxKVjw0szLt7Zlzj3xX4t/sa3F5b/tPeAJtPd4y2r2q7kJU4LgHOCOv+1xX7pfsC+BfFHxO/4JK+PPAvgq2a91PUbiZLeBCAWIkzgZ78V+ff7Mf/AATt/bI8H/H7wd4q8TeCL22tdP1K3luJ5WTCxxyAknB/hFY4LGUKbxcars7/AKHfmWExNT6jKktLHqX/AAXObH7VmlfxN/ZEH3ep5Oc19Ef8FNY93/BNP4OI204S3Vd3TmA5xn23V85/8F0JI/8AhqzS4PlLRaRC3ynkYcjB/wB6vvT9s74A/Fz4/f8ABO34QeHvg7or63eWltbTvBFjIHkEZOT71z06sYYXAupsn+h0wp1J4zMVS35f1P5YvJjj2tGFCBhtXHp6HvX9Ef8AwSv/AOTA/jRIu4ELN83/AGzNfkn8Qf2FP2rvhT4Xn8aeOvCF1p2m2a7p7lygVR68En/x2v1s/wCCVrR/8MA/GYNwSs33STn92fUV73E2Io18GnRd9V+Z8jwhQrUsxl7f+VmN/wAEk/FFn8b/AIA/Ej9jnxQVKXVnJdWSSc8nkgA+h21+EHjjwjqHgHxlqvg/VkaOfS7iSBw3TMZIxj14r6q/4J7/ABuk+Av7WfhzxlNL5VndXX2O6XsY5TjB/ErX0x/wWK+BafC39qKTxxoUKrpfi2Jb+Fl6GQgb/bl8mpyyKwuZ+z6TVzTN5vHZSp/bg7B/wRUbd+3lpf8A2CL3/wBASvlX9v8A/wCT2viYPXXJv519T/8ABE9v+M8NMZm/5hF9/wCgJXyz/wAFAtzftsfEwKVCnXpvmbkdfT1qsM3/AG3Vc/5f1OfGa8OUk+58eSqyt5bfKVb8sd6/oS/4JFyMv7Kfxp2jYv2CQqvX/lkecf8AstfmD8Pv2Efi78Sv2a9Q/am0O404aBpYl8+KUkTYhJBOAhB5H96v0+/4JEyNJ+yv8aZm+VjpzMF9jEcYquJ8ZTrYN8nRr9DXgvAVcPjE5dYs/NX/AIJzmRf23vBcmfvakdzDjPJr7E/b0X/jblBH8u77Vp+5u/QdPmxXx/8A8E5+f23PBaj/AKCh/ma+vP2/GjH/AAVxt5jtVVvNP78c461yVv8AkYtL+R/mj1cJZ5Wnv+8X6n6If8FIPh9/wTs8SfF7Sr79qjxT4h0XxCulxqkWkmMRCLHBO+J/mzu/ir4X/bG/ai/Y1X9inS/2Xf2a9X1LVlsrgOj38f7wDHJdwAD/AN8ivTv+CvX7MHx4+Mnx80fxN8MfDN1rNjFpESNPAAwyc8DOOma8R+PX7EPgH4Mf8E9dH+KHiTQpdI8cyzBLhp5Oc4yBsxx/31Xi5TRoQhh51ajeux7+c4nFOpiIUaSStueq/wDBGstH8D/jMu5hiy+YN/1zNfglrSyNrV5t3EfaJN2BxjJ61+/H/BFOzutY+FXxl0XTU869ubMKiZySWjIAH4mvxZ+NXwK+K3wZ1z/i5uhT6Q168j2/njHmrvOSMeh3Yr6nKaiWZ113Ph87ot5Ph3DY8WVdvA4pKdI0jMTMzMf73tTa+wPzSQUUUUEhX+it+xF/yZh8If8AsStG/wDSCGv86mv9Fb9iL/kzD4Q/9iVo3/pBDX5d4o/7tR9T968C/wDfMR/hX5n1FRRRX4gf00FFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAH/9D+/iiiigAooooAKKKKACiiigD8c/23IfL+Nzzf89bKH9M1xfw7k3eG4o/9o/zNex/t56QbX4gaTrLfdvLRk/79sP8AGvAvhtdbtP8AJz8oYr/X+tfwD4l0XSz/ABkZLXmv9+p/S2TT9rkGHa7floepUUUV+enKFFFFABXzz+1xe6hpv7K/xI1LSZnt7m38M6g6SoSGUi3c5BHII+8rL0r6GrkfiB4RtfiF4B13wDfbfI1zTp7B93TbPGYzn/gJruy+tGliadWa0TX5mVeEp02l1P4Ifgr+1j+0R8BNci8TfDLxXqFnMcM8UspmikHUh0kyMHvtwfev38/Zl/4LreDNcjg8O/tQaK2j3nCtqumgvAx6ZeIklB6tvI9q/mp8deE9U8B+NNX8F61Aba70u8lt3icYI2vgAj6bTXK1/oXxJ4fZHxBTU8VS1e0o6P7+vzufh+AzzGYGVqctF0P9D74Z/G74Q/GTRV8Q/C3xJp2uWbqGV7WUHr2ILZz/AMBr1Kv84XQNe17wvqUWueGb2fT7yP5knt5CkgPsRgivqfQv2+v20vD1qtpY/EvXpIgu1fPupJCB6Aknp29K/B8z+jhXU74HFpr+8v8AK/6H2eG49g1++pW9D+9GV44VaSZlQD7zMQB+Zr4x/aC/4KCfsp/s2280Pj7xTBPqUf3dN04ie4J/65hgAM8Md3Ffxi+PP2tP2nvidavpvjzx7rmpWki7Wt5buUwn/tnkivnmu/Jfo5RVRSzPE3XaK/V/5GGL48bVsPCz8z9uP2oP+C2/xy+J/n+G/gHar4K0l8hbtiJr1x6hyAiAj+HYSPWvBf8Agl/8Q/iX4k/4KDeCbjVdd1G9mv5bv7R580kgkH2aYkEE4xxuVduAelfmDX7Sf8EN/hPdeMP2qrz4nTW7Gz8Iaa7pOw4E848oJn1KOW+gr9O4k4dybhzhnGrCUVFcjXm21ZXb13Z89gMfisdmFL2sru6P67n+9TaKK/gE/agooooAKKKKAPHfijJua0j/ALrH+Qr7a/4J+W5XQfE1z/C1xAq/gj/418H/ABGvPNvoofRS35nH/stfo/8AsJaO1p8Mb7WP4by8IX/tmMf1r9l8FaDqcR0ZpbKT/Br9TPjWXs+H3B9WvzufcNfmB/wWV/5RufEn/uGf+nS0r9P6/MD/AILK/wDKNz4k/wDcM/8ATpaV/enD/wDyM8P/AIl+Z/K/F3/Ilxn+CX5M/hGooor+pz+Cyb7sIb+EN8wXr04I+hr8l9c8UaL8Df8AgpZ4Z+MHxMgZ9BtdWs712YZAij2AuBz0IZsV+sW3cvsa8N+OXwA8E/HTRRY+Jv3N3b5WC6TGVJ7Edx/s181xJllTGUk6W6P0PgHiKjlmLk6/wSVj9+Pjt+3V+zncfCu88dSeJtObRWs2ZFWRCZcjgBAWOTnG3Ffw0/D34S+Pvj18TvEXiD4PytpFpJcS3CsrmILGz5CcY/2fl9q+trX/AIJx6+14LPVPErtpqNu2MHwR7Atj8lr9D/hT8IfBvwf8Mjwz4Qi2xNhnkODJI47k9hXzTybFZnWgsVHlhE/Ra3F2XZLQqf2fPnqS8tj8sLz9in9qT7HKf+EjeTKndH9okJIxk8dOa9M/4JX/ABa8F/s5/HTXvDvxWcWN9exeRbzy4UCQEggucYz/AHi3Nfqh8yqrqfmDfMq818P/ALQX7E/gn4xasfFmj3n9k6oRtZ1Bw3JPQd/evJ4o8PfbYa2GWp9D4Z+OcstzVYnGpJHuP/BTL9pL4U6l8C7/AMOtqNrqOpaq4+z28BDlMZ5yNxGP9r1r56/4J56PqWm/AhLu8bKXFxI6KxBIBJ59ea8L0/8A4JpXF0slx4j8RvcSlCIlwWOe2c54ryuTSf2mv2Nda8zS99/okbFmVQZISM5OcdDj6V89whw7ieH63tasT9C8W/ELDeIa9lhaiTP26oZmZvavjX4H/tofDn4qJFpOtN/ZGqthdkpAVj7E/wAq+yIpo5EE0bq6FdyspyCPbHNfuODzfDYuPtKTufxvmvDuOwNT2eJjZkyqpprLtrg9Q+J3w90m+XS9S1i3iueFZNwBz/n8q7K1vbPUoUurGZJonx86kEEex710xxlGWikcE8vxNNJuL1Las0bbl4rC8TQXF54fv7e22pK9vIqtnGCQcZJrzf4qfHj4d/CPT3m8VX8XnbTsgUgyH6D/AOtX5i+L/wBoz45/tKak/hX4UWc9lYFiP3QJIB4+dx2/Kvn844gw9Om6K1bPs+GeDsZiZLFT92C6s9C/4Jg/Gj4d/An9pLxNafEp0s57/wA2K3vH4AYEkjPYH+tffX/BS79pL4V6l8CdS8Oy6ha3up6vhbW3gIYjGfn46YzX536H/wAE09U1Tw22peKteNprD/O2ASATzye5P6VR1D/gmr4gl0u7urrxC1zeRofs6SgnJHbJ6A1+B5h4e4+tiljOXQ/t3IvpB5dl2S1MkjJO+lz6G/4J26DqWj/AxrjUFZIr64MsSsOcDPI9Otfp18Nfih8QPg74uj8cfDLVJdK1OFfkniPJzwQc8Ecn71fh3+zP+0Jr/wACteb4L/FxGhsI5Skcr8GIk9CTxgmv0o1j48fCHQZorfWNes4pZF3KqnJ5GcAe9ft3DuZYOGBVCo7W3TP4443yXMJ5tLF0VdS1TR+rX/D0b9uz+Lx24x/0xj/wr57+Nn7UHx2/aOazj+NPiCXWI7LPkIw2quep2Dgk4HzFa+a9B8RaL4msxqGg3SX0PVmi54+vr7VtNt6xt/u+tfSYTLMKrVaKWh8Hj82x7XssTJn1B8Hv2zv2mv2f/Dcngv4Q+Jn0XTXlM7QRRgjcckkk85yfpXqzf8FP/wBu6TlvHk//AAKGM/z4r4JVs/xfL978TS/N93NVPKcJUlzzpq5NPP8AHUo+zhVdj0H4mfFL4g/GTxdc+OPiVqUuqanOo3yuQThegHYICWYAepr6a8L/APBRr9tDwR4bs/Cvhfxi9pp9hGIIIlijIVQMAAkdhXxL820qvQ0MrrtYbt1a1svwtSmqU1oYYfNsVQqOcJWbPsD4hft7/tZfFrwrc+AfiF4tl1HSb1QlxA0SAEHjrjPFeXfDf9o741fCHwbqvw/8Aaw2naLrYK3tuoEgfIweTnGRXiHzf0qNPubegaphl2GhHljDQqpm2Kqy55S1J4prq1ukvoSqSxOHVlzwQcg59iN1e3/FT9o340/HDStN0f4ray2qw6OmyzVkAMajjGR8/wCdeFqzLt2/w9KN23Lbuv3q6Z0KTs2tUc0cVVinBStc9K+FHxc+InwL8ZJ8QPhfqX9l61FE0EdwFDERyABxghhyAvWud8ceMfEXxC8WX/jjxhdNd6rqUpuLidgBukPU4HAzXKj5srnr96nbmFHsKal7TqQ8TVcVTvoe/eFP2oPjt4N+FN38D/D3iB7bwtqDH7RZLGMEyHL84yMksflrN+Gf7RHxk+DvhvV/B/w51htM03Xk8q9gVQfMByCMkZAxXifO7/aojOM8/wAJX5ueD1rKWBpWdkbf2lX0al5HX+B/Hnif4Z+MLbx54Ju2sdVsJBLbzqMlDzkgDIPWt34g/GD4hfFfx2/xQ8falJea9LtZrpQIyTH9wgDoR26ZrzUNj5o+Pl2/hUattzx1+WtI4WN+ZIj6/V9n7O5936T/AMFMv24tD0230iw8dXC29sgjT5ATgDAzkZJHqa8i+Mn7XP7R37QWn2+h/GDxLPqttC29IGAjXPqQnGfrXzev97+L71Nrio5RhYTc4RVzsqZ7jqtP2UqjaPaPgz+0F8X/ANn3WLnX/g3rUukzXKBZ9nIcdgQeCRx9K0vjV+098cv2jLqwvPjLrj6y2mKVt2aMRlRkkj5Pc14K3zMP++lqRn6bT/nrXR9Qoup7VrUw/tOuqXsVN2G56Kp3Bfu02iiuuR5cgoooqSQr/RW/Yi/5Mw+EP/YlaN/6QQ1/nU1/orfsRf8AJmHwh/7ErRv/AEghr8u8Uf8AdqPqfvXgX/vmI/wr8z6iooor8QP6aCiiigAooooAKKKKACiiigAooooAKKKKACiiigD/0f7+KKKKACiiigAooooAKKKKAPgL9vXw2154L0nxVGN32K5MDewlGc/mm38a/O7wfrEGlxz/AGo4VMOvqT0wK/af49eDZPHfwn1nw/bgGYw+bFns0Z3ce5AI/Gvwg2srFZAwYfKyt2Nfxx445O8PnaxPSrFfetP8j+hPDjFRxWVPCzesX+ev+Z6xdfFS4Z8Wtsqp/tnJP5ba6LQ/iJpeoyC3vg1tIflVmOQT9e1cv4S8D2OtaT/aF+5DOxVVXtjvXO+KPB914dYTKfNty20PjkH0NfiHJDY+v9hg5S9itz6OpGZVUyyMoUfxNXjHgnxuLW3bS9YdiiKWR29ux/8AZa4vXvEepeI7z94zeU3ypEvT24HU1n7F3Zw0spqOq4PY+lIry1uP+PeZXx/dNWK+UZrXVNHkS4mR7c/eVsEfka9w8D+Km1y3NnfN/pMK/e9R6/WidK2qdxYrLHSjzwd0fy//APBbT9km8+HXxai/aa8KwMdE8WuE1FlHEN6B1JHAEw6f7h61+Ftf6JHxc+EvgX44fDvUvhf8SLJb7SNViMUqMOQT0dD2cH5lIr+UH4jf8EpfiJ8BfjdKvitG1fwFCxuLPUUGfNGfkilA5R1+XflcHnBPGf7C8IfFHC1cuWWZhK1Wkvd/vLovVbH5PmHBGKx2ZwpYFfxH9z8/I/P34Q/s1+PPixGuqQhdN0wt/wAfU4Jz/uJwT/30Kr/Hr4D6l8FdStdtw19p94u1LjGMSAcoevP8Q9q/aO1t7eztY7WzRYo4lCoq8AAdAK53xh4L8M+PNFbw/wCLLVbu0dg21uoIOQQexr9DhxfX+s801an2P37E/R+yxZPLD0Jf7V/O9r9rdvvZ+VHwZ/ZZ174teFLjxXcXy6ZCzbbPepbzCOpI3DA7K3fn0ryX4mfB7xx8J9SWz8VW2IZG2xXCcxyfQ9j6jtX7laVptjo+nw6TpcSwW0ChERBgADsBWb4o8F6P8QtHl8H6xa/bIr1diooy2T0KAc5B+Zcc5op8X1liHOf8P8hZh9H3Lp5VClhp2xMV8XST810Xofz9wwzXEyW9uGeWRgiKOSSTgAf7xr+3T/gmH+yfcfsp/s2Wun+JIvL8SeJmXUtUUjBjJH7uI9wY0OD7joK+Bf8Agnf/AMEkZvh58QP+F5ftCIlwumz79D0tuc45E9wDxkfdVDznJOOM/wBCVfgvjb4lUMztlGXSvTWsn3fRfI/DOGOGquCqSrYpe+tAqrJeWcMnlzSoCf4Sa858eeMLjT2/sfS2xIy7pXXqAew968fjsdS1CN7qGJ5QPmZlBNfzrCnfW9j9QwmVOpBTqOx9XK275lOVP92lZljUySlUULuZm6AV85eFfFl5od0IZnY2jNtkRuce4/umtbxx4wOqTnS9Nf8A0ZG+Zl/iP+Apexd7Clk9T2vs76HVat8TbGzkMOmxNPj5d5OB+HrWPb/FCSTMN9bqob5dyHp+HOaw/C/gWfXI1vr5vJt/4eOT9Pb3qbxt4Ps/D9vDeWLsUdtjKeucE/0rTlp7dTvhQwcZKjbU53xNqEN/qhkhbcgUKrevf+Zr9pP2YvDZ8MfBTRLWUYe5i+1MvcGXmvxh8F+Gbzxl4u0/wrp4/eX9wsCt6ZOMn2Ff0IafY2+nWENhbKEihQIqjoABiv6Q8AcpvisRj3tFcq+er/L8T858UsYqeGo4OPr9xoV+YH/BZX/lG58Sf+4Z/wCnS0r9P6/MD/gsr/yjc+JP/cM/9OlpX9f8P/8AIzw/+Jfmfzjxd/yJcZ/gl+TP4RqKKK/qc/gsKcrdd3Q/eptFO/2SuYNse5dqDA+7xz+ef/Zacnl/dK53U2il0sEpd2OYrv3R/L/s07zG/i4X/Z4qOigfMxQq7twXJ/2uaqahY6fq8Bs9WhS5jf5SjgEEd+KtUVE4KekldGtOvUpvni7M/Pn41/sF+E/FksniD4Yz/wBk6hy6oo/dsewGNpBr4P8AFnxe/ag+AOn3Pwh8RXM8a3qhIHflgOmYz71+/DrHuMjbmbb/AA4//VX5if8ABSD4beItW03QPiZ4eia4/sdis/cgZBQnHYYbP1r864r4fWGwssVgfdZ+5eHvF88djKeAzW04d30Op+Gv/BIP4hfED4RwfFLxz4pbTdT1S2W8t7dsnAkAIEhzkHH3h2NfC83xE/aM/Zq8Zap+zzHdPNeGcW6q2TgnBBTP98FW/Gv1y+GH/BWr4T6v8H9PsfiQ1xZarpVqkE1ui5EkkYAJBHGCRu21+cPgPVvEH7XH7ah+M1pZtbaZaXkdx8w4EcIGASO5x096/DOGcRmtbHujzM/tTxOyzhHCcPUsVhYL2lr7t/mzvPhb+wv4q8dalF4w+PF/Ogkbe1v1kJ6gEk8A/Sv048F/D/wf8PdNXSfCFilnGFC7kAyfcnHJrsm8zu2d+dyt+n5U2v6fyzIMNg0pcvvvqf51cQcYY/MJWcrU1slsOXrhufx/nTmbbDtjbLLnarc8HrUbfL04pyn5tvZvlZu3Ne7y730R8apO90flx/wUn0Twjp/hjS9dFuseszOUSZeMqBhs8dc4xX3X+xH/AMG+6ftBfs9af8avjV4vu9GvNdtRdafa267tsbDKFySPmYFTivh7/gpZL4fufB2lQTXQXVbSUlIFOTh+ckdsEDrX6ZfsL/8ABfn4RfDH9nHSfg/8fdOvY9S8OW62sM9qAVmWMAJnkEEBVB+XtX8feNdbNqVR/wBjvXySZ/e3gXSy+rl0f7V/E7fSf+CYviD9hf4d3mvX3iZvEEF3eGJUUECNcDYcEn5jzmuTVmCr838PHGOK+LfE3/BRD9pX9uz9qxLrwrNdaV8PbeUMumr/AKsRJ1JOcFzy1faHzH7xVs/hX7L4N1M1llK/tV3n8l+R/PPjthMupZ1fLlZMdlfSm0UV+uc2x+FzbTdg+b+E4ob5utFFPmFzDt2dvtSq21fm+ZjTKKUpBzBRRRRLUOZhRRRQ5XDmYUUUU+ZhzD8r6UyiipDmYfN/DRRRRLUfMwpSxakooFzBRRRQSFFFFABX+it+xF/yZh8If+xK0b/0ghr/ADqa/wBFb9iL/kzD4Q/9iVo3/pBDX5d4o/7tR9T968C/98xH+FfmfUVFFFfiB/TQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAf/S/v4ooooAKKKKACiiigAr81P2p/2nNe0vXZ/hv8PJ/swtlxeXSYLFiNxRDyAMHBb1yO1fpLJu2N5fXBx9a/nt8dLcL471ldSVvNF/Nv3dc+Yc1+JeNfEeLy7L6WHwj5faN3a7Lp8z9L8Nsmw2MxdSrXV+ToRL408bLcfa11e93lt2Wmcg/h0/4DXMTSSTObiY7nclmb1JOSa+nNJutKutPjtVRPL2hVXAwRXK+IPhxY3ym40fbBJ97Z/Af8K/jyeNnU0qO5+00czpRlZx5TD+HeviFDpcp+6xZfcH/CvXbq3tdRtTb3AV45V2svsa+Wrqz1DQ77y7pGhmiO5f/rHvXsng/wAWLqEYhmO2UfeX+o/9mrnqQ6oxzPB6+3pHk/iLQ5tC1J7GTlPvI3qDXTfDWxhudceaZc/Z03Krep4/SvUPF+gr4i0kiHaZovnRv5j8a8X8H6odF10eb8gf5G3dj6H8aqM+eDOqliXXwsktz3jxHpKa1pMtlIuWClkZuoI54r558P6k2h61DeScKjbXHseDX03a3Ud0u6PqPvLXg3xC0BtL1Y30I/c3PzewPcfj96ooy3TOLKa29Cp1Pfo2jdRJGdwZdwb2NQ3dnZ6jZvY6nCk9vIuJEcAgj3B4rz34d+IFvrH+x7hv3tuvy7u6/wD1q9KrN3g/M8mvRlQqch8geOv2Jfgz4une+0tJ9GuHbc32UjyyT3IcH8gRXhF7/wAE6pmkLaf4qRE3fKrwEnH1DqK/TaivqsHxrnGGjy067frZ/mfRYXjPN8PHlp1nZd9fzPzj0L/gnhoUEgbxJ4jluE/iW3jEZ/N9/wD6DX1h8N/2efhR8KmF14Z01ZLpf+Xq4/eSH354B91UV7ZRXLmXFeaY6Lp16za+78jmzDinMsXH2des7fd+Q4tu+9VW+uo9PtZb64OEjUs34dvxqxXj/wASfECjboNqemGl2/oP/Zq+ehG7PHweGdeqqZ5pNJca1qzueZLh/wCfT9K+ldC0+HStPS2iG1eP8mvJvhzoP2q8OrXA/dRfKu7ua9rkmht43mlOEC7mb2ras9bdj083r3aox6Hzj42sYdP8SXEduFRHw6qvbPX9ab4P8OyeIdSCyLi3h+Z2/p+NV9evJPEXiKWa2DfvWCovsOK9+8NaHDoelxWqr85+Z29Saqc2oHoYvFvD4ZJv3zcjijhjSGMKFC7VVewrxH4la9DeXEWjWrZW3+Z29+mPwrpvGnjFdNU6XprbrgrhmXsP8a8f0nR9S168+z2KM7H5mZug9yamnD7bOPLMHyf7TVY3RNc1Xw/fDVNDna2uFUhXTggEYOD2+vWtqLxx46tbpNSh1a9WQNuV/OkPP4nFeraB4B0nS1E18q3Nx/eboPoP/Zq1vFUmkw6HcQ3yrjbtVfQ9sfjXXDHVado020bVcwoVKtlTufXv7Kn7SOqfECY/D/xzL5upxoXt7jABlQckMBxuUfxdx2zyfMf+Cyv/ACjc+JP/AHDP/TpaV8wfs6R6hJ8bfDY0/cXF4jPt/wCeYI3/AIYr6f8A+Cyv/KNz4k/9wz/06Wlf3T9HziPF5pSorGO7hOKv3Wh/NfjjlGHwOFxf1dWUqcnb5M/hGooor/QQ/wAzAooooAKKKKACiiigAooooAKp32nWuqafLpuqRJcwXGVkRhkHPqKuVR1bWNJ8P6bJrWsXCW1nCpknllPAA9+3WscV7P2b9psehg/ayqJUFqfhf+1t8JvBvhP9pPRPC/h61FpZak0LTInT946A49OCa/abwD8OfBfwz0NNC8FWKWduVDNt6k9Mk9T0r8Sv2pfjf4J8bftCaN4w8N3D3FhpRhWWcjBJjkQvgd+AcfNz7V+0fwr+LfgP4saGNW8D3i3KIoV0YYkBPqM8V+bcMzwf9p1/ZteR+5cd0sy/sPCOpfRanpn3WoknjjVmyAsa5bd0A7mnJ94NnBWvH/jxrl7ofwl17UdLP78WpCsvbPGfwr9IxWJ9hRdQ/Ectwf1nExo33Pmr41/t2+Cfhpq0vh/w3AuqXcJ2u7H92DnoMYyfxryK3/4KXaDdeH7xrjR/K1NRtgVCfLye5Byf1r9FP+CAP7LfwD+KGn+K/jx8Yra31nWNOnEVvFdgSJGDgl9h7nJG5qb/AMF3v2Z/gb4d8F6V8bPAulWuhaq16bd0twEE0YAIOBxX5bXx+Z1cNUxsZWS6H9E0OHMioYunls6V59z8nfgT8EfG37TnjJ/iv8VHlOmK+5d+R5vfAz2+lfoVrX7I/wACdduIri40OJJEXaqxZAOPXDDNWP2TtUuNX+AOgXd2qo3lBflGBgD09/6V9IeZ932r6nJMgw06CqzjzN9z844r4wx8Me6NB8ijokji/Bvw88H+AdPOm+FbCKyh27WVBgsfUnrx9a7baFULuUkLtyveo2bdTa+so0Y0lyU1ZH5ti8bVxM/aVndsXb/F/DSUUVrKRxX8wooopAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFf6K37EX/JmHwh/wCxK0b/ANIIa/zqa/0Vv2Iv+TMPhD/2JWjf+kENfl3ij/u1H1P3rwL/AN8xH+FfmfUVFFFfiB/TQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAf/0/7+KKKKACiiigAooooAo3Nzb2Vs95dOEjiUu7NwABySfpX4a/tC+LPBPjT4n3/iDwNC8drLhZXOMSSDgug28BsL9evfj9Ov2udcvdD+B+pfYW2NduluzDsGOT+eNv41+P3hDS9P1XVBb6kW8pV3MqnBNfy/48cRt1aWUJaL3m+vVaH7X4Y5ZGnSqZlJ+RBofiS70lhH/rYS3zK3b6GvZtG8UWt9GrQvvVeqt1FZ998N9AvIfM093hbb1zkfl2rzXVvC+veGZPtYDFEb5ZU5H49xX80y5Km25+kzeGxj00Z7pquk6T4mtfJu/mZejLwQa8H1rQdW8J3yybmxu3JKvQ+x9/aug0PxyyMIdS+Vv4XX+or1iOSx16za1ulWRHX5vQ+496mLdN67HPCdbBStPWBz/g3xMNWQWsnEg+8vv6j2rhfiJoX9n6gurWoxFcN823s3f8+tbkPgPWNF1yK+0WVWiDfxHBA7g+teoX1jaalbvaXyK8T/AHg1K6Urw2MniaeHxCq0ndM8j+H/AInkbUE0m+Od67Ubvnrg16jr2kQ65psunzcM/wAyt6EdD/jWTpPgjQ9HvhfWoJlT7u45A/8Ar11tROavdHJjMTTdbnonyukmoeGtY3fNFPbt/n8DX0doOvWfiCxW8tWw33XTuDWD408Jrr1v9qteLtPu+49D/SvFdF1i+8Nan5yhgQ210buO4Na6VFc9ecI5hS518aPqKiq1jeW+o2cV7atuSRcr/hVmuY+ZcWm0woorJ1zVbfQ9NfULj5sLtVfU9hQVCDm0luzL8VeJrfw7YlvlNxKu1E/qfavA9M0++8SasLdTulmbc7t2Hcmm3VxqXiTVvNl3SzTHaqr2HoPQCvoDwv4Zt/Dtjt63Dr87/wBB7V1/w15n0nuZfR/vM3NN0+DS7GKxtV2pGu36+prxPxx4wuL66l0ezbZbxsVZl6sR/SveK4u88B+H76+N9IjBnbcVU4BP0rCEkndnlYDEUoVfaVtTj/hz4e8zdr14vyq22JW9e5rpPGvi1dHhOn2JzdOvzN/dHr/veld1Fbpb2otrNQiou1V7CvJdP+HmoXmoPfeJJVOWLMoOST7+gquZN3ZvCtTrVnWrvRdDk/DPhO+8TXDXUzMlvu3O7dSfQZ6mverGx07RbMWtqqxRD7zdyfc9zRdXFjoen+Y22KGFdqr0FeK6946ur5mj0/hfu726/gO1P3qnoXJ1sdKyXunomv8AjKz0tTGrfP2C9T/hXiOsa/fa1N5lwcJ2Velbmi+C9c15vtUwaOI/M0r9T9B96vSrP4b+HbaH/St8x/iZjgflVR5KfmzrhPC4R23Z1X7JHjPwD4M+JS3Xi9GjuLtRBZ3DH93EW4OR/DnpnOBXrv8AwWWIb/gm18Rz1B/sz/06WlfCniLT7fS9auLG1OY0YbW+oB/SvpX/AIKS65feI/8Agj/4n1nUmZ7i4sdILs3Un+07MZ/Gv7B+jNxE6mOjlLVlGUZJ/NXufhXj7lUVlFfMYv4oSX4Ox/E9RRRX+mx/laFFFFVysrlYUUU7+EtUy0DlG0UUVXKx8rCiin4X1qfIgZX5cf8ABR7xtrSx+HPhnpczQw6o+6XnALEgIT9Mmv1Hr4h/be/Z11340+D7PWPBYU61o7l0ToZYzyQD13ggY/GvmOMsNXrZZUjh9z9C8OMVhcPnFKWK2P3j/Zp/4Ie/sR+Fv2aNGsvidoCeJPE2t6bDcX9/OcvHLMgJ8okcAE4DK3vX8zfjz4Qz/sIf8FI7/wCAfhG+e60Q3kcEBkOSYbgAgPz1TfgN7Zr6t+DP/BeD9rT4A/CSy+B/jbwpZ6/qejW4sLLUb8yxTrHENiI8YKhygAUsysTjnJr5h+CPw/8AjT+0d+0bfftafHkv513P9sBlUxmSQACMIhAwiKFH3cHHc1/Jvh7w5xJSzydTEt8lz+3vErP+H/8AV/ki1ex+qTsGY8/Kfun+VZPiTQbHxLot54f1Lm3vIjE24dj71sYVc8YXcWXb2zjFJ95WjP3f4cdcV/arpKVLkqI/zwjXdOt7Wm9j8jtAsf2vP2C/iLfeLv2fZpzp96xVlQeZFIDyBJGSM4+6tQ+Orz9tD9v7xpY33xule30ywxtVl8mCME87EHc/7v41+u3/AI8P7rAfnnGaGx6tt/u9vy6V8f8A6oLmdqj5H0P1GPijW9lz+xXtf5jlfA/hbTfA/hex8K6Om2CzgCL6nAxk+jV1ki9BTf3fzyMuCflXb0xTf/Qa+yo0fZJQgflWLxDr1XWluwoooqpaHJyhRShS1J3+7xVcrDlYUUUUcrDlYUUUUcocrCinbfl3U2pDlYUU5l202jzDlYUUuPlLf3aSjyDlYUUUUByhRRRVcrDlYUUUUcrDlYUUUUcrDlYUUUVJIV/orfsRf8mYfCH/ALErRv8A0ghr/Opr/RW/Yi/5Mw+EP/YlaN/6QQ1+XeKP+7UfU/evAv8A3zEf4V+Z9RUUUV+IH9NBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB//U/v4ooooAKKKKACiiigDyz4w+ENN8efDfV/DmpyLBHNAXErdFMfzg/pz7V+DEc1xpd8zWsgLRkqrjkHBxkeoNfth+1PfXtj8C9fm07KuYkVmXqFMgB/w/Gvxh8MWtjdatFFqC7ouW25xkjoK/kvx9rUv7UoQUffUNX31dl8tT948LlKOArTm/cvt8jvdI8b2rqI7o+RL/AHux/wAK9Bt9YjuIv3gWVHX7y8gisa68E+F9WT/RU8h/4WTsfcd6881Lwv4m8LsbmxZpIR/En9RX89Wi9tz7P2eGxDtHRnZ614E0vWFN3oZW3m6snY/h2/lXPeEIfEXh3xAml3kLeVLlW7ge4NR+HfHH+mRRaoNm5gvmL7+or3BW3cjmhtrR6kYmtWoJ0quqYUUUVgeGFFFFABXjfxL0OOFotct1x5jeXIq+vY17JXnPxGuVOivDn7rD88j+laU5e8ellk2q6sZfwt1JpI59JkOVHzx/j1/9lr1qvnXwH5q6yZI2YbUPzL74r137bef89G/OitH3zozPDfvm0dbXhvxO1SSbVItKX7luoZl9SRnP5V6L9tvP+ejfnXiHi7c2uzeZuycH5vTFVRWtzTKcN++uzvPh3pccONSkGXk+57Af417BXm/hm4VdLtZo/uhf6mvRo2VlWReV6rUTlqcOZTlOq2x1FFFQeeFOXls02igDwnWLPxR4w1iW3WJhDG5Vd3CgDvn1P3q7nQ/A+k6HGLu+K3Ey/wATdAfYf413c0kcMLTTFQEXdu9AK8R8QfECbUJhb6PFgDKqzck/QV0KTlotD3KFWtiUqVJWR6hea9b2sfmZVEX+JjgV5rrXjteY7FmkP3dzcAfQd6qaV4I8QeIJBd6w7Rxt/E/JI9h/jtr0GLwX4V02HE0Pmkr8zOcn8PSlyxXmxqOFw7s9WeI6Dpv/AAk3iaz0u8uUtvts6xPPKcIoY4JPsK+zP+Ct3h7TvCv/AAS48d+HNLGLexg0qCLvwup2YFfFOtQ2dvq1xDYsxhV/l3f59a+sv+Cn15qWo/8ABJHxVfaszNczWWjtIW65OpWWc1/V/wBF+tS/tqVPl9+8NfK+x+Q/SDhJ8P1KkX7nJPT5H8RNFFFf6lH+UA5dufmoVlfnPH3akjyzRwx7mkdwEVQSxY8AADqT0xX7Nfs0/wDBMvwpa+DrX4qfttT6jo8Grpu03QdNJ+3Sg8mV8fMiIPnYc5A9eK8vMs3oYJc9Vn0GTZHXzCTp0Vsfi+22M/e4o+Ztu1d2V3L7j1r+l34Ff8EVfgzrV14u8R+PNZuvEnhFbeO68M3+m3RgJUiQyxzoA2HTCAqT+HPH8115CljqE9vu3iNyvzdwuQAPp3rDKeIsPj5VFS6HXnfC2KyynGeJ6ldmU7VVfm5+X6dahVgzbV+avpb9kz9nHVv2rvjto/wW0u8Wwjvd09xeOufKhiQyOcDqcAhAWGTjOK+1NS0//gkHaeILn4Wyp4wR7eVrN9caUGETIShcpnPkhwx259a0xecxpVvYJNswwWR1KtD27dl5n5Lq2P8AvrbTtqj+LpXT+OdN8M6T4x1TSfBeovrGj21xJFZXs8expogeJMZOM+mTXPSR3SQiaaJsS/cZwRn6EgBv+A16sKvPZ7HjTpOE2iL+IL/E38NNzuU/LuZ/lC5wSPY0/dIy7m3Y4VmbPXtk44prQyLCk1xGyI67lZlIBHqCeCf92r0+0tyIRe6WxlzaHod1Mbq6sbWeT72940JOO5JBbP8AebPWtKNdkaqoUL/dXoP6fguBX1T+y/8As1n9pLUPGNq2rvpcXhLw5PrzOkfmGXySAIuSNmScE818oWtwt1Ckjc70DNu7cDjH9d3vXDQ+re2lCkrTR6eJeJdGMpv3GWlVd33l3U3a3WneVdpbi4mibymbarsDj8CcD/x6jy5GG8jj7zN1J9zxwM/L9TXbzN6tnl8r0uNZSvWnNuVdzdD7U7ZdLEt95LCIttV8HGe4zjGfo1NaT5fMypG3buYdPaq06PQnkaI6M+/6VYkintwklxC6I/zKzAgEexKj9KjVpJGW3j3yOfuogJb8B3ovDuPl6Ef8Xl/xf3adtfdt/wBrb+NSXCrC5juE2MG2tuGMH3HUH1qS3W1jnia+ybcSx+aqcEgHkA9uM7fepnN2uxqKbS2Kq/K+4f8AfNIJd3zblxX7E/A/wv8A8EmvjJ8RND+FdjoPjmHV9anjt1eW4AhEp75DE9f7ora+O3gb/gkz8B/i1rfwZ8Q6H45n1PQp/Ile3uA0bOUD8EsCRyP4a+ZnxGlW9j7J3/rzPr48Lt4ZYrnVj8Ym6imNuX/9VWpLfzry4t7KB5YgzMqMCSI8nZkAcEDbuJ702GKe+3R2sbzMy8tECSD1wQMn/gVfSKSsmfJTpcjsVaDx1p0XeMD5gwXjPB6YI69akjjuriQwwwPJIjfMiAkgjuQBkD8Krm0IULuxDuVvunNFO82Rc/MzqP7vAB7g1My3UdqbvY4tz8u5QQufTOMf+PVco2HyMhdf4ugWnMNq/N8uK+2v2F/2afB/7R3xE1m++J11PZ+EvCWmtqurNanEpjjBISMjpvw249vevGPjx4m+BPiDxRBcfAHw5e+G9Lt0MUv2248+SYggB8bF2HG7Prn2rzI5gp4n2CWx6s8pnTwqxEtL7HhFSJ93pwf6U/ybhUF3JEwiLfLI0ZAP0P3TSKskyt5YJO75uvJHPIA4AG3c1ejo7s8qUNkHy7eeBSLGGYqvZd1KsczKbiNHMYYK0qg4B7AHp/X2qPbJIJfkcmL5nVQcqfU8dB905xzUR31YuRoUOv8ADz+FMXc3yr8zfeqxBFdXTO0YeYqu5tiliB0BAGcDu1Qs6t95ejfMuOD75rXlh/MHK0N3fpTtvf8Ah+n/ANlTliupoXuIYmaOL7zKDgfiAR+dOjWSbIX5vl+bvjIznA9uaOWH8wSi1YhoqZY5pkMixNtVsM+CY8jsT90HH3adFHdXERuoY2eNF+ZlBwPqQMD/AL6rPmQ3Bor7f4sUURt5kZYn8MZP5/LRVc0TKUbBX+it+xF/yZh8If8AsStG/wDSCGv86mv9Fb9iL/kzD4Q/9iVo3/pBDX5b4o/7tR9T948C/wDfMR/hX5n1FRRRX4gf00FFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAH/9X+/iiiigAooooAKKKKAOd8ReH9O8T6HdeHtUTzba8QxOvse/1B5r8PvjH8GfFHwa8Svpt/E72UrlrO8UHEiZ4BI6OB94fjX7x1z3iLwzoPizSpdE8SWkd5aTfeilGQa/OPEHw+o8SUE0+WtHZ/o/I+w4T4sq5PVel6b3X6o/BjR/GkluFj1Ibtvyq69fx9a9Q03xJHeR7reRZl/i9R9a+kPib+wuXaXU/hZehRy32O6J4/3Hw34KQPrXw54r+Hvj74e3zW3ibTbiwdPmV2GV+ocbh/49X8f8R8DZrk9S2MotLutY/efuOAzPK82V8LU1fTr9x6pJpPhXUJluru2VZA27dyBkfTArrVaM/6tlZf9mvm2w8Zapb4EzLOP9rr+ddVZ+OtPbCzB4m/vLyP518bKnL1OjE5ZW01ue1UV59Z+K7G4XbDdK3+y3H86149eZl+WVG/EVlyW3R508HUW6OqormG1qZf4kX8qzrrxFHGrNdXCov8W0/0GaSjchYWo2dXeX0dqpHV/wCFf8a8G8cax9tulsY2ysbZdvc/4Vc1rxku1rfS9xZvvO39K3Pg18G/Enxm8TjSNNLQWic3V2RkRj/2YnoFz9cDmvWyrLK+LxMcNQhectkezQo08FTeKxDskcb4Y1XTdHEsl0W3vgLtGcAV2CeMNBb/AJaMv4GvqDWP2A/FcLH+wddt5x/enUxn9A9clN+wn8YI+Ib7S5V/2ZJAf1jr7PEeGHEMH72Ek35Wf5Hm/wCs2SYh8/1jX+vI8LbxhoH99z+BrhPEmpWOq3QurPd93a27jOOlfWsP7CHxab/W6hpaL6K8hP8A6L/rXa6P+wDrEi/8VB4hit2/6d4i/wDMpV4Xws4jqO0cI/nZfmJcU5Hh/f8Ab/r+h8c+CdUVozpc3UfMn07ivVtP1BbceTJ93+FvSvJPiL8O/FHwm8XSeHddQxvC26CdQdsidnB/mOx/W7pPjK1mjWLVP3bj+MdD/hXwWPwFbD1pUq0bTW6PWxNCGJgq9F3gz29WUruj+Yf7NLXn9vqVrIvmW86/8BIFaS6pN/z1Vv8AayK4eVniSwjTOuorj5NaZf8AWXEY/EVj3XizT4vlmutzf3Vyf5Uoxb6FQwdR7HoVwsLwmO52lHXayt0INcvZ6X4Z0WTzLG3Uv97dyfyJzivP7zxzZrn7LEzn+83Ark77xdq11lQ6wq38K8H862jTl6HpYfLK2q6HtGq+LLexUtNKsf8As9T+VeU6140utQzDZ7og33mbkn/CtTwT8IfiV8SLry/DOlzThm+aZxtjAPck4U/hmvvX4W/sOaJpMkerfFC6XUZlw32ODIiB93OC4/AV9zwz4fZtnM19Upe5/M9F/wAH5HFmGb5VlC/f1Lz7df69T5P/AGevgJrfxd8SQ39/C8Og2sgNxO2QJMcmND3LdyOnWvb/APgsbbQWv/BNT4iWtuqpHGNLVQOAANUtMV+mem6XYaLYxaZpMKW9tCoVI4xgADsAK/NX/gsr/wAo3PiT/wBwz/06Wlf254RcCYbhuvQpQfNUlOPM/n+SP5q8V+J62cZZi6k1aChOy+TP4Rqm+7iRRvDVDTlZl+n92v7kP85Yn7ff8Et/Af7Ofnab4gsVi8XfGDVbqWDS9EvEP2TT0j3kXcpI2OMDeoXLg4wPT+pr4SfA3T/h/dT+M/FUza94u1FAt5q04BIA58i3Q8QwDsiAAnLkZJNfxh/8EofiL4F+Fv7aGg+LviJqtro2mRwTo95eSLFEpMTgb3dgAD0DE9cDvX7Ef8FiP20vAfiD4H6Da/s5/Eiyu9Vj1LfPFoeoI03l4GC4ifOzP97ivxPivLMTXzN0o3tI/pbgHPcJhso9vNK8dD9cvH3wf8RfC+7v/iZ8BYkS2uFaTV/DjER212uCXkiAwkNzjPIwH43ngV/FP+2l4Z/Zl0f4lxeIf2Xdamv9F1uA3lxZTq4ksZiQfIzIMv1b5gTjGM1/W58Hf23v2YY/2adJ0vxd8TvDw1j+xAk63GoQCbzShBDgvnfn+E85r+HTVpI21S7mhHBlkdQ2cgE4yc9CSVb1r0OA8BWhiantOn4nj+KGZ4athqPsdb6+h9B/sm/tHa1+yl8eNF+M2j239oLp+6C4t2OPNhmBRx1wTtJZM8A46da/TGb4D/8ABPP9u7xBNN+zv4nuvh7481xpJ00HUgTBNOTkgP8AOgy3REOT2FfmV+zD4P8A2ffHnxCuPCv7RWvXXhuwntzFZahagGOO4OQDLkOBCPlZm7Y64r9EPgz+yH+zj+zp8VNH+P3xC+OnhjWtE8J3g1K3s9IkEl3cNHyiBBI5JPRgq59MV9NxD7KFd1INqp5dT4jhuVWVBUqtnS/Iz/2Bf2I47n46fET/AIXlokOsS/CtHQ6TLLHHDPfE4i3mRxCYThvvnHTrX1d8JvDf7SXxg8bTfDX9rvwB4LtPh9rEE0Cz2EukQzaY2CYngkt5POJR8KV7jORXyT8Ff26Ph74i/ae+Kl98ZPP0fwZ8Wkkt3nQEyWbAg27yAdQBuz0POc4Brg7z9iL4ceGZLvXvE37Q/hweG7dWlR7K5M91MADgJbpMzh89Rj14ryK1PE1Kv792bStv+B9Fh6+Ep0v9lV7N9j2/9jXwn8Kfh/8As1/HrXPH/hmy8XR+DtRL2aXGCJDbu/lEyDJ2HClwrHI4wa7f9kH486P+3vD4x/Zr+N3grQbSFNDuNU0m602zit5LPyCgAjMSL13jnOeK+Zf2XfiF8N/Cf7EPx28F6t4hsrTUNWUrplvdTJFPdKAcFI5Dkkrgn5SQTXM/8EofH3gn4bftEa54g+ImsWeg2U3hLULVJ7+ZIYzM5hKIHkYKXIDbV6nBxW2MwNX2derrzrb7kc2FzSlCVCgrcj3PrT/gkn8aL3TfhD8VPC40LRpv7A8M3eopdy2kRmnYkOY7iQgvJCCcbHYjGBjivH/2LdB0f/goN+0nb+LvjB4Z0iy0XwPohvLjS9GgisYbggu8QcRiNDl1wxdgCMAnFee/8Eu/il8O/CPiL4hfDv4latb+HIfGvhq406zvb1hHCk8hAAckgAHH8WPQVrfs6+Ivhz/wT7/aGTwn4w8YaX4z8K+LdEk03XLzw/J5q26SF40AId95TO846A4xkVnXwtSFbEezXvtfoawx9KdHDqVvZp6/efc/gW3/AGpvGXxXTwT8dPhh4Gi+FOozyWc9nby6NG1lAeBLHLHN5xkQAH5Tk5rwz9g/4N/Cnwj+0h8fPh74n0u18X+HPDGjXrWf2jEgmit5UMZEgyoJA+ZlOea8P1b9iH4ajXLvxJD+0N4ai8KO5uElW7L3LIST5YtxNv8AMAKrjH4Vd/4Jz/EL4V/DHxx8aLXWPFNnHYah4X1Cy0u81GQW5uzkCPYkhGHcAkJtZ6j2Evq03Sl22TB4ql9apKqtr9j3H9hv9rjTP2qPjPa/swfEz4feGYPBPiazuILOCztIFks/LiLoUlRFd3yBl3Oc85zXjP7Gv7P/AMM/CniL4y/HX4saWviPR/g1BJPZ6TOMw3U5eTyw4PBCBF+U8HPNfNf/AATB8XeEfh/+2N4M8WeOdQtdF0uzimWe6vJBDFGTAQMudoBJ43FuvHWvqL9m/wDaM+DeifF742fAn4vaotn4L+LbSWCavEd0cMgL+XICMgI+/kngYByM12YzBVaXtoUb2svz1/A5sBmGGxDpVK1r3f5O34nzz8QP+Chnjb4keGdc8H+OPBPhS50u/tzHZpBZQQSaeSQQYpY4w+QAR8zAc/Sv0S+H/wCzn8R/2eP2P/B3jL9nfwlofiD4h+M1F/e6trklkRbW7AmJLdL1wMkbQxAx1Oc1+fvjb9iL4UfDzw3q3iLxZ8a/DN9Z2cEn2C30Zhd3FxIcY/dxu7gY4bIOCRzX0Boni/4W/ttfsr+D/hXr3jmz+H3j74dobWCXUpjDaXdoSRGokLoA65Hykk5GenFTj1CdKP1X4L67kZdUarVFivj6bG5+118LvEHjL9jt/jN8dPD+g+FfiXoGrR2jHRJrILqFnMQPNkitHIDoS/bJwK+a7f8A4Jt/FC88JjxdH4y8JJEbc3SwNqERkAxnGN/BwPz4rz39oL9n3wL8Gfh/BdTfFfT/ABl4nuZht0nSJHuIVjz87yXAd0BC/MoOM9hXxa+7cfKLJub5lU8fT+7ivYyrDVvY/uqml+z/AMz5/OsZQ9v+9p627n11+wKwb9sj4eTKzAHUlbHHoa+jP2nLK11D/grVfaZqCCW2uPF9ijo4BUgiEEEHhgR1DV8u/sU+IvD/AIR/aq8D+KPFF5Bp+m2Goq9xcXEgjjjGCMlzwBuPOa99+PnxB8E61/wVEuPiNpOrWdzoJ8VWVx/aMTpJb+SohBkEoOwoCDkqcDFGNoS+vy0+ydOW4mKy+nFv7Z9f/t5ftNeCP2Zvjp4w+DXwJ+H+hxnUFM+rajdW8bTGSdAHjgBUhIUAUqFYck8V8r/AH9qD4weHfh/o3g/9lz4PadfXWnKV1TVl086lNeyZ5DyvC/kgtuBRWUDOB0ryv/gpH4w8IfEL9sPxT4s8G6jb6zpVzLGY7uylEkcgAGQJEJBPuvSv0V+Ld9p/xS+CfgK1/Zd+MHh/4eeErLRIoNX05buOzvIrpUHmPIEdJpnODgKck1486HssHRTWr9ex61HF/WMdXadkvTucl+3J8HPB/h/9pT4GfFLTfDEXhu58btp8ms6QkQjhF2JQJMxYUAHGCMc9SMk1u/tnftaaf+yX+1Pq/gH4F/DzwzaR+fDcapcXlrFNJdGRADGPMQiGMAfKqMBkk45rkP2qvjJ8HfF2pfs4/wDCG+NIPENt4fuoINRvbqXEymK5IeW4EhLxoSCQXIOzBJPWvj3/AIKYeMvB3xA/bA8QeK/A+p2+tabcCLZdWUgliYAAHEiEg8jnH6Vll+Eq1nThVu9H37ndmeYUqPPKlZO6/I+tv2yfhL+zv/w1Z8H/AIlXVjB4a8I/E6zg1DVLWBfLhhIKEkBMABy4D/LgDrxXsv7YGv8A7VnwX8Qaw3hP4Z+F9W+DjoyaX/Z2n211CYdmBKZYkeQED5i5xggkHFfLX7bXiT4O/Ga1/Z88H2/jPTrSztPDX2LVbyJhcCxkIhA+0RxkkHIwVbBHJ6CvoL9lOJv2MfGV9468dfHPw/4k+HNtZTJLodlepdyagGiKRRxWxdzDhypZVUcAg8ZrGXtadKE27tdHcpV6NSVSnH1voVP+CS/xw1a1+GPxe8MtomkPaaJoMmo27y2kRkkMhkJjlfZmaEY2qj5AGeK8n/ZRh8JftpfHHVfi18bvDOkWXh74d6RNqMunaPbxWkV1LGR5ccqRIAQec5XP4E1yn/BP74pfDXwvH8edQ8V6vZeH4fEOiXC6al5KIfMMskzpFGHYZcAgYHr0rx3/AIJ7/tH+CfgD8UtZ0P4sM8PhTxnp0ul3s8YMhgMpGyUgLkhPmU+mc9q6KmAfNXqwTvp/wTmhm1N0qFGpJcmp758Jf+CiC/Fn4zaR8Nvit8O/C9z4B8SXsenDSrewt4WtI7txHGYp0QPlGdWzuU8ZFe0fsq/ss/D/AOFf/BUrxl+zzrFrHqnh6y0288j7aolJhmtxIDggglN+Ax54zXjPwv8A2Of2c/gr8VLD4v8AxA+NnhnU/CPhu8XUoINOmje9uBA4kiQRByd+8KH+XkZworvv2Tf2svBHxI/4KaeKv2jvHmo2vhrRtWsruKB7+UQxrGIBHEMucZcBW29ckj2rHFQb5/qafJy/idGFxNOPL9dtfm022Pm3x5+19ofxQ1rSf2ffAHgbSPDXg6y8QwNarFChu3EMpTNxKRly+Tnc5zx+H3Z+2T+0p4Z/Zd/bqm+D/wAOfh14aTRbu4spNZa4tIppLsXeDImXQ+Sq54VGAJAJFfhT4JvLW1+MGn6hdyrHajWxP5rHKiMTk5JHGCOc+lffX/BSr4ieBfiH/wAFB7nxx4H1my1bRXbTHW9s5UmhbyggfEiEp8nOfm4wc16UspXt40dbcr+/Q82jnN8JKtpz86+7U+tP2vvjr4L/AOCf/wAa7D4e/s6+BNEfTNSt/wC3NWOp28U8k4uiXFujuCYYUQqAqNgY6Yrjv2rP2RPh/wDED/gop8Pvhn4Btk0PQviLZ2d7dQ2ahVgMu8zmMAAKCAuFXHevmH/gqp4+8D/Ej4+afr3w51az1mzHh63glnspkmjEioAUyhYZH3WC19I/tjftQeDfAv7Xnwd+Ovwz1Sy8RJ4Q0Sye6WynEozHvDxSeWTghTlgeRxmuClha1KMJ073cX+R31syw1arOFS3IpL89T6Y+LGnftRfDX4vT/Df9nP4Y+DIvhlok4svsd7JozzXsMR2PLK883nB3HPz4IJ5wa4H4X/s9/CXwb/wV58IeHdB03TpvDfiXSLvUn0jdFdw28rWZ82AgF0ISbcw7AAY4xXi3xw/Z1+Df7SPxS1T48fBX436Douj+K7h9SuNO1m6+z3Fm0pzIhiMiHGT8owCMYrhP2J9Q+CvwD/4KX+Hr5fH1vrHhnSrW7SXxBen7PbiRrQoR5kpxsaQsEOeeOprlhQq/VpNPXlff+rnRWxdH6zG8E482mxw/wC0l+1v4Tkh139mD4R+DdI0LwZDqTWq3vkob6YRykyPJKQzZySFwxwgAHTFfo1+0ta/HT4N6fo+rfsU/Dvw54n+DcOmwMz29naahLcNjMhnLh5yT3bB9znFfz8+PpNL1b4pazdR3SLZ3er3DrdIPMURvK5EgA++CDlWHWv10/Zh+Gt5+zj8TtG+IXgz9orw8PAtrie6iS8QSSQkE+R9geRiJCPl2qmeK9XMcAqFKlUpv5a6/wDBPKy/NVia1aEtr76aH44+Ntaj8UeNNU8QLp0Gjtd3Uk7WduCI4SxJ8tEIUoEJ2KuB07VytfQH7UvxA8H/ABX/AGiPFfxC+H9qtjomp6jK9mir5ZeMkkPswCN/3yrdCcV8/wBfcYRt0YNn5rmUVGvKwV/orfsRf8mYfCH/ALErRv8A0ghr/Opr/Sn+EHhjRPBHwn8L+D/DMH2bTdJ0i0sbSAszeXDDCscabnLM21QBktk9yTX5h4oz/c0Idbs/cfAmi/rWJqeS/X/I9Rooor8VP6UCiiigAooooAKKKKACiiigAooooAKKKKACiiigD//W/v4ooooAKKKKACiiigAooooAKy9Q03TtWtTZapBFdQt95JVDKfqCCK1KKJQU1aa0LjJpprRo+V/HH7Inwc8YM1xa2T6Rcvyz2ZwP++DlB+Ar5e8TfsFeJ7TfN4U1iG6UfdinBVv++h8v/jtfqTRX5/m/hhw/j3z1MPyvvHT8tD6rL+Ns2wdlCtdLvqfh/rn7Kfx00PPm6Mblf71uwlyPoMmvOrv4T/FPSjtu/D2pQH/ahcf0r+gWmbQ3XmvgsV4BZdN/uMRJLzs/8j6uh4q4xfxaUX96P57F8E+Pnby10m9J/u7HP9K6zQ/gH8ZPEEwjsfDl6uf454zGv5lQP/Hq/ePyof7q/lU9c1D6P+ET/e4lv0SX+ZtV8V8S17lBfeflx8Of2Ftfvplu/iTeizhVtzW9uQzn238qB+Ga/RXwd4H8MeAdFi0DwlZpZ2sf8K9SfUk5JP1NdlRX6vwvwNlWRr/Y4e9/M9X/AF6HwedcTY/M3bEVNO3QKKKK+vPnQooooA80+Ivwv8H/ABQ0Y6L4vs1nVcmKUcSRn1RhyPp0Pevzo8ffsOeOtHme48CXSarbbiVilISUD3J+Un6LX6xUV8RxR4fZRnnv4unap/MtH/wfmfT5HxZmGWLkw8rw7PY/AvVvgj8YNBkK33hy/Cj7zxwuV/MLiudXwR8QFbyf7Kvdx/h2Sf4V/QqQD1qLyouu1fyr8urfR/wjd6WLdvRf5o+4peK9e3v0F95+Atn8Ifixq0gFn4b1KX/aWFyPxNenaH+yV8dNcAk/slbRG6m4kEZH4Hmv2wVVXpTq7cL4B5ZB/v68n6WX+Zz1/FXHP+FSivvZ+Yfhf9gfVpisvjHW0jQ/ejtUyw/Fsr+lfVvgj9lf4OeB9lxFpv8AaFyn/LW7PmZ/4Dwn/jtfR9FfoGT+G2QZc1UpYdN95a/mfJZhxlmuM0q1tPLT8ilbWltYW62tlGsMScKiDAA9gKu0UV95GKVklZHyzlcK/MD/AILK/wDKNz4k/wDcM/8ATpaV+n9fmB/wWV/5RufEn/uGf+nS0r2OH/8AkZ4f/EvzPnuLv+RLjP8ABL8mfwjUUUV/U5/BYMq+XtbkH5WXsR703y41X92AD/eX+lOopciu9DVVpJW6DZEhb+BeVw3Tn9KkfywoyOvzMvU89RnvzTaKcEobaBKtJjm2tH5a/KAwZe/1H41H5NvxhVwPurjoPr9TTqKLa3tqL2krWuOUlW/dHGfmYdefx9ahjhjVfLbcV3btvYHOcjipKKlwWmhUa0ujDbG0m6QfxbvUj1+vH0o2qV8uQK6n5m3DPNFFVZdtyfaTHMqhl7gZ29iPQ/xd+aBHhV+fLbtzNgdfUD17U2ilFLoio4iS6ieXCysp3YdtxHQZPce9PlVZseYPuL8q9SCO4PvTaKOVbJBKtN7u41o1bIkVT/Fu9O/T+L607au7KsuSu1twz160UU2k3sKVZ6DVjXd+85A+VdvBx/8ArFOlQSMJGOWHzbscknn9DyKKKXItrDliJN3vqESrGyr8pX7rE9SD1zUqtx0x/s+n0qKiqt20JlVbd2O2qykE4/iVvcc1CsEbKY8YG4ts6+/XjvUlFTKOtwjUaBFWNVj+XC/eVRin+XGp8xTlj8zN6+mfXFMoocVbYPaSTvsLFFCrMxChvvM2Mgke3b+7940ke2PHlrgBvu+o9Cf/ALGiijlQ5Vpjtscausfy5+8y8E46ZPP8qj8mPjaq7hyGbk5/TrTqKHBdivbTCWOOTHCqy/dZhkjPWhvmX5dqnd93HpRRStHsR7V6eQLHGGVv+Bfj6D0x6/pQyxsq+Z8+M7Vbnr6GiijkSVkhyrTDb8pX5cbtwX69/wD61HlwhWjjC7W+XbjFFFX1T6h7WQLFDGrKoC9Plx1A9aBFGuFjAwudvbk980UUrL+UUqrG+SsjFm27mbc20dfrTmWNlEe1SoxuVhnJB6/gPz9qKKlQSZXtp3HKqr/Cp+YN9Mf4/wAVQtArLtIXd/FtGAfcj5uf9qpKKqy/lBVpq4L8pbbtAYBdvbg9evWiiijZIzlK4V/pn+CP+RL0f/ryh/8ARYr/ADMK/wBM/wAEf8iXo/8A15Q/+ixX5J4p/Bh/WX6H9C+BHxYz/t39TqaKKK/Gj+jQooooAKKKKACiiigAooooAKKKKACiiigAooooA//X/v4ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK8k+Mvwa+HH7QXw41H4RfFzTjq3hzVjF9stPNmh8zyZknj/eQvHIMSIh+Vx0wcjIr1uitaVWVOSlB2aM61GnVpulVV0z8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6K9P/WDMf8AoIl97PD/ANUcm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6P8AhzV/wTb/AOibf+VPVP8A5Lr9P6KP9YMx/wCgiX3sP9Ucm/6A4f8AgK/yPzA/4c1f8E2/+ibf+VPVP/kuj/hzV/wTb/6Jt/5U9U/+S6/T+ij/AFgzH/oIl97D/VHJv+gOH/gK/wAj8wP+HNX/AATb/wCibf8AlT1T/wCS6/SywsbXTrCHT7JfLigQRIuScBRgDJ54rTorlxWYYnE29vUcrd3c9HAZPg8DzfVKMYX7JL8gooorhO8KKKKACiiigAooooAKKKKACiiigAooooAKKKKAP//Q/v4ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD/0f7+KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA/9L+/iiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP/T/v4ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD/1P7+KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA/9k=" } }, "cell_type": "markdown", "id": "263d8baf", "metadata": {}, "source": [ "## Background \n", "\n", "The Transformer deep learning architecture has proven very successful, and has spawned several state of the art model families. One among them is Bidirectional Encoder Representations from Transformers (BERT): 340 million parameters [1]\n", "\n", "With transformers, the “pretrain then fine-tune” recipe has emerged as the standard approach of applying BERT to specific downstream tasks such as classification, sequence labeling, information retrieval and ranking. Typically, we start with a “base” pretrained transformer model such as the BERTBase and BERTLarge checkpoints directly downloadable from **SBERT** or the Hugging Face Transformers library. This model is then fine-tuned on task-specific labeled data drawn from the same distribution as the target task.\n", "\n", "\n", "\n", "Information retrieval (search) systems use lexical search algoritms such as BM-25, TF-IDF to find answers matching to a query. When we are able to use pre-trained language models like BERT for search systems, we can achieve higher search relevance as the pre-trained models will help in finding **semantic matches** rather then just **term match** for a query. At the same time, one should consider fine-tuning the original BERT model before using it for specific downstream task like information retrieval which helps in curriculum learning. \n", "\n", "The SBERT framework which is based on PyTorch and Transformers, offers a large collection of pre-trained models tuned for various tasks. We will be focussing on fine tuning the BERT model on data retrieval (search) usecase.\n", "\n", "In this notebook, we are attempting to fine-tune the BERT model for information retrieval usecase based on the original research paper Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks [2].\n", "\n", "**References**\n", "\n", "- [1] “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding“, Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova.\n", "- [2] Reimers, N., & Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084." ] }, { "cell_type": "markdown", "id": "c09b2282", "metadata": {}, "source": [ "## 1. Development environment and permissions\n", "\n", "Lets start with setting up the development environment and permissions, First we make sure that the kernel is set to \"conda_amazonei_pytorch_latest_p36\". Once the kernel is ready, we start with installing and importing all the required libraries." ] }, { "cell_type": "markdown", "id": "9af7c7cb", "metadata": {}, "source": [ "### Install and import dependencies" ] }, { "cell_type": "code", "execution_count": null, "id": "5b257eb2", "metadata": {}, "outputs": [], "source": [ "import subprocess\n", "import sys\n", "\n", "def install(package):\n", " subprocess.check_call([sys.executable, \"-q\", \"-m\", \"pip\", \"install\", package])\n", " \n", "install('sentence_transformers')\n", "install('opensearch-py')\n", "install('requests_aws4auth')\n", "\n", "import json\n", "import requests\n", "import boto3\n", "from torch.utils.data import DataLoader\n", "from sentence_transformers import SentenceTransformer, LoggingHandler, util, models, evaluation, losses, InputExample\n", "import logging\n", "from datetime import datetime\n", "import gzip\n", "import os\n", "import tarfile\n", "from collections import defaultdict\n", "from torch.utils.data import IterableDataset\n", "import tqdm\n", "from torch.utils.data import Dataset\n", "import random\n", "import pickle\n", "import argparse\n", "import sagemaker\n", "from sagemaker.pytorch import PyTorch\n", "from sagemaker import get_execution_role" ] }, { "cell_type": "markdown", "id": "9434966e", "metadata": {}, "source": [ "### Setup the Sagemaker session, region and IAM role \n", "\n", "This notebook is already configured with an execution role which gives sagemaker, the permissions on behalf of us to access other services like S3, Sagemaker model training, sagemaker endpoints etc.\n", "\n", "We have created a S3 bucket for this notebook to store all the model artifacts. In the following code, we save the execution role arn and s3 bucket name as variables to be used later. " ] }, { "cell_type": "code", "execution_count": null, "id": "f4785d94", "metadata": {}, "outputs": [], "source": [ "role = get_execution_role()\n", "account = role.split('::')[1].split(':')[0]\n", "bucket = \"sagemaker-nlp-\"+account\n", "boto3_session = boto3.session.Session()\n", "my_region = boto3_session.region_name\n", "output_path = \"s3://\"+bucket+\"/nlp-dualencoder\"\n", "output_path" ] }, { "cell_type": "markdown", "id": "c52a530a", "metadata": {}, "source": [ "## 2. Training\n", "For model training, we are using Sagemaker Pytorch framework and provide a custom training script (nlp_loader_test.py). This script does the following steps, " ] }, { "cell_type": "markdown", "id": "5f110b64", "metadata": {}, "source": [ "### Downloading the data\n", "\n", "We are using MS MARCO dataset (https://microsoft.github.io/msmarco/Datasets). This is a large dataset to train models for information retrieval. It consists of about 500k real search queries from Bing search engine with the relevant text passages in descending order of relevance that answers the query.\n", "\n", "The dataset has 2 attributes,\n", "\n", "
Attribute | \n", "Type | \n", "Description | \n", "
---|---|---|
Query | \n", "Text | \n", "The question asked in the search engine | \n", "
Passage(s) | \n", "Array of texts | \n", "The responses that the user voted as relevant to the query they asked, these responses are ordered in such a way that the highly relevant response comes first and the responses with poor/no relevance comes last | \n", "