|
0
|
94
|
new file mode 100644
|
|
...
|
...
|
@@ -0,0 +1,1292 @@
|
|
|
1
|
+<!DOCTYPE html>
|
|
|
2
|
+<!-- Generated by pkgdown: do not edit by hand --><html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><meta name="description" content="ClassifyR"><title>An Introduction to **ClassifyR** • ClassifyR</title><script src="../deps/jquery-3.6.0/jquery-3.6.0.min.js"></script><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><link href="../deps/bootstrap-5.1.3/bootstrap.min.css" rel="stylesheet"><script src="../deps/bootstrap-5.1.3/bootstrap.bundle.min.js"></script><!-- Font Awesome icons --><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css" integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous"><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/v4-shims.min.css" integrity="sha256-wZjR52fzng1pJHwx4aV2AO3yyTOXrcDW7jBpJtTwVxw=" crossorigin="anonymous"><!-- bootstrap-toc --><script src="https://cdn.rawgit.com/afeld/bootstrap-toc/v1.0.1/dist/bootstrap-toc.min.js"></script><!-- headroom.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><!-- clipboard.js --><script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><!-- search --><script src="https://cdnjs.cloudflare.com/ajax/libs/fuse.js/6.4.6/fuse.js" integrity="sha512-zv6Ywkjyktsohkbp9bb45V6tEMoWhzFzXis+LrMehmJZZSys19Yxf1dopHx7WzIKxr5tK2dVcYmaCk2uqdjF4A==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/autocomplete.js/0.38.0/autocomplete.jquery.min.js" integrity="sha512-GU9ayf+66Xx2TmpxqJpliWbT5PiGYxpaG8rfnBEk1LL8l1KGkRShhngwdXK1UgqhAzWpZHSiYPc09/NwDQIGyg==" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mark.js/8.11.1/mark.min.js" integrity="sha512-5CYOlHXGh6QpOFA/TeTylKLWfB3ftPsde7AnmhuitiTX4K5SqCLBeKro6sPS8ilsz1Q4NRx3v8Ko2IBiszzdww==" crossorigin="anonymous"></script><!-- pkgdown --><script src="../pkgdown.js"></script><meta property="og:title" content="An Introduction to **ClassifyR**"><meta property="og:description" content="ClassifyR"><!-- mathjax --><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>
|
|
|
3
|
+<script src="https://oss.maxcdn.com/html5shiv/3.7.3/html5shiv.min.js"></script>
|
|
|
4
|
+<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
|
|
|
5
|
+<![endif]--></head><body>
|
|
|
6
|
+ <a href="#main" class="visually-hidden-focusable">Skip to contents</a>
|
|
|
7
|
+
|
|
|
8
|
+
|
|
|
9
|
+ <nav class="navbar fixed-top navbar-light navbar-expand-lg bg-light"><div class="container">
|
|
|
10
|
+
|
|
|
11
|
+ <a class="navbar-brand me-2" href="../index.html">ClassifyR</a>
|
|
|
12
|
+
|
|
|
13
|
+ <small class="nav-text text-muted me-auto" data-bs-toggle="tooltip" data-bs-placement="bottom" title="">3.3.1</small>
|
|
|
14
|
+
|
|
|
15
|
+
|
|
|
16
|
+ <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbar" aria-controls="navbar" aria-expanded="false" aria-label="Toggle navigation">
|
|
|
17
|
+ <span class="navbar-toggler-icon"></span>
|
|
|
18
|
+ </button>
|
|
|
19
|
+
|
|
|
20
|
+ <div id="navbar" class="collapse navbar-collapse ms-3">
|
|
|
21
|
+ <ul class="navbar-nav me-auto"><li class="active nav-item">
|
|
|
22
|
+ <a class="nav-link" href="../articles/ClassifyR.html">Get started</a>
|
|
|
23
|
+</li>
|
|
|
24
|
+<li class="nav-item">
|
|
|
25
|
+ <a class="nav-link" href="../reference/index.html">Reference</a>
|
|
|
26
|
+</li>
|
|
|
27
|
+<li class="nav-item dropdown">
|
|
|
28
|
+ <a href="#" class="nav-link dropdown-toggle" data-bs-toggle="dropdown" role="button" aria-expanded="false" aria-haspopup="true" id="dropdown-articles">Articles</a>
|
|
|
29
|
+ <div class="dropdown-menu" aria-labelledby="dropdown-articles">
|
|
|
30
|
+ <a class="dropdown-item" href="../articles/DevelopersGuide.html">**ClassifyR** Developer's Guide</a>
|
|
|
31
|
+ <a class="dropdown-item" href="../articles/incorporateNew.html">Creating a Wrapper for New Functionality and Registering It</a>
|
|
|
32
|
+ <a class="dropdown-item" href="../articles/introduction.html">Introduction to the Concepts of ClassifyR</a>
|
|
|
33
|
+ <a class="dropdown-item" href="../articles/multiViewMethods.html">Multi-view Methods for Modelling of Multiple Data Views</a>
|
|
|
34
|
+ <a class="dropdown-item" href="../articles/performanceEvaluation.html">Performance Evaluation of Fitted Models</a>
|
|
|
35
|
+ </div>
|
|
|
36
|
+</li>
|
|
|
37
|
+ </ul><form class="form-inline my-2 my-lg-0" role="search">
|
|
|
38
|
+ <input type="search" class="form-control me-sm-2" aria-label="Toggle navigation" name="search-input" data-search-index="../search.json" id="search-input" placeholder="Search for" autocomplete="off"></form>
|
|
|
39
|
+
|
|
|
40
|
+ <ul class="navbar-nav"></ul></div>
|
|
|
41
|
+
|
|
|
42
|
+
|
|
|
43
|
+ </div>
|
|
|
44
|
+</nav><div class="container template-article">
|
|
|
45
|
+
|
|
|
46
|
+
|
|
|
47
|
+
|
|
|
48
|
+
|
|
|
49
|
+<div class="row">
|
|
|
50
|
+ <main id="main" class="col-md-9"><div class="page-header">
|
|
|
51
|
+ <img src="" class="logo" alt=""><h1>An Introduction to **ClassifyR**</h1>
|
|
|
52
|
+ <h4 data-toc-skip class="author">Dario Strbenac,
|
|
|
53
|
+Ellis Patrick, Graham Mann, Jean Yang, John Ormerod <br> The University
|
|
|
54
|
+of Sydney, Australia.</h4>
|
|
|
55
|
+
|
|
|
56
|
+
|
|
|
57
|
+
|
|
|
58
|
+ <div class="d-none name"><code>ClassifyR.Rmd</code></div>
|
|
|
59
|
+ </div>
|
|
|
60
|
+
|
|
|
61
|
+
|
|
|
62
|
+
|
|
|
63
|
+<div id="installation" class="section level2">
|
|
|
64
|
+<h2>Installation</h2>
|
|
|
65
|
+<p>Typically, each feature selection method or classifier originates
|
|
|
66
|
+from a different R package, which <strong>ClassifyR</strong> provides a
|
|
|
67
|
+wrapper around. By default, only high-performance t-test/F-test and
|
|
|
68
|
+random forest are installed. If you intend to compare between numerous
|
|
|
69
|
+different modelling methods, you should install all suggested packages
|
|
|
70
|
+at once by using the command
|
|
|
71
|
+<code>BiocManager::install("ClassifyR", dependencies = TRUE)</code>.
|
|
|
72
|
+This will take a few minutes, particularly on Linux, because each
|
|
|
73
|
+package will be compiled from source code.</p>
|
|
|
74
|
+</div>
|
|
|
75
|
+<div id="overview" class="section level2">
|
|
|
76
|
+<h2>Overview</h2>
|
|
|
77
|
+<p><strong>ClassifyR</strong> provides a structured pipeline for
|
|
|
78
|
+cross-validated classification. Classification is viewed in terms of
|
|
|
79
|
+four stages, data transformation, feature selection, classifier
|
|
|
80
|
+training, and prediction. The driver functions <em>crossValidate</em>
|
|
|
81
|
+and <em>runTests</em> implements varieties of cross-validation. They
|
|
|
82
|
+are:</p>
|
|
|
83
|
+<ul>
|
|
|
84
|
+<li>Permutation of the order of samples followed by k-fold
|
|
|
85
|
+cross-validation (runTests only)</li>
|
|
|
86
|
+<li>Repeated x% test set cross-validation</li>
|
|
|
87
|
+<li>leave-k-out cross-validation</li>
|
|
|
88
|
+</ul>
|
|
|
89
|
+<p>Driver functions can use parallel processing capabilities in R to
|
|
|
90
|
+speed up cross-validations when many CPUs are available. The output of
|
|
|
91
|
+the driver functions is a <em>ClassifyResult</em> object which can be
|
|
|
92
|
+directly used by the performance evaluation functions. The process of
|
|
|
93
|
+classification is summarised by a flowchart.</p>
|
|
|
94
|
+<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAvIAAAFPCAIAAACgRb0tAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAHIPSURBVHhe7Z35e1XV9f/5Byr480cRse3TWurYVkFkEEVbtVa/Cm3V1rGArYozKE5VEe1jbZ2oE0VFHKAqgiMggyAghDGMSTPPhgwmIQPVfl+567B7cocQcs9Jcg/v15PnPuess/bea+999l7ve+690O+/QgghhBCRQLJGCCGEEBFBskYIIYQQEUGyRgghhBARQbJGCCGEEBFBskYIIYQQEUGyRgghhBARQbJGCCGEEBFBskYIIYQQEUGyRgghhBARQbJGCCGEEBFBskYIIYQQEUGyRgghhBARQbJGCCGEEBFBsiZIvhVCZCDeAhZCZD6SNcHAzvjNN99UXffzstGD9Kc//WXQH8uWxStxI0Q0kKwJANM0ra2tbJFff/11fX19XQwOhBB9EP8KZdmyeKVshIgGkjUBwG7Y1tbW0NDA/lhRUVFeXs6rEKKPw1IFli2LlyUsWSNEBJCsCQDe57W0tNTU1LA/FhUVlZaWStkI0cdhkbJUWbAsWxYvS5iF7C1pIUTGIlkTAOyG+/bt++qrr9gfS0pKqqqqamtr9SGUEH0WlieLlKXKgmXZsnhZwpI1QkQAyZoA8Msa3gWyaTY3N7cKIfowLFKWKgtWskaIKCFZEwB+WcP7v6ampv3792P8VgjRJ2F5skhZqixYyRohooRkTQD4ZY3tj//5z3/YOr3LQog+BsuTRRq3bCVrhIgAkjUBkChrtD8K0cfRshUikkjWBID2RyEyDi1bISKJZE0AaH8UIuPQshUikkjWBID2RyEyDi1bISKJZE0AaH8UIuPQshUikkjWBID2RyEyDi1bISKJZE0AaH8UIuPQshUikkjWBID2RyEyDi1bISKJZE0AaH88nKmurh4/fny/GDNmzPCsos+jZStEJJGsCYAI7I/ffvutJWbHcccdN27cuLlz59Ijz6n3SAzPz8qVK3vr33Sm3bFjx3px9Ov36KOPMvW9FYw4JCRrhIgkkjUBkOn7I2n4P//5j5eZEzjyyCMXLVrUi6m68/Dgs88+6xUxQYt79uyxGFavXl1fX9/U1NTW1tb3Z5/ICXLq1KlE/pOf/KQvKNeeJ9OXrRAiKcHImn//+9+2uRuPPfaYd6EL4Pz555/b8bnnnjtp0iQ7DhZqBu8kaDJ6fzTRQMw2d++8805xcXFpaenSpUtfeOGFk08+2ewLFy7shm5gzCmbzkczLrza2tqKigpimz9/PnWOHDmyqKiopKSkqqoKPdHa2trzY06LKCqCGTVqVGFhIbEx+/YfnXZjrMKGWSBUZsQ0DSN25513Yjn11FMrKyv7YMBhc9Bly5gchsMiRKYTgKxBl7A5vvHGG945lcZ2T++kUyiFM6rIOw+N44477pDE1iFx0P2xL8PGTYarqalhImD27Nlbt27Nzs7euXPnnj178vPzf//732MfMGAAqqLru3wsI3gf0KTz0Qyl2traEC4oLeIhtlmzZlHn0KFDN23atH379ry8PPtf01E/3Wuie1hgn3zyCcEMHz5827Ztu3btQmYRKvaejOSgEAwwC4TKjDBQzPjXX3/NvcrQoRebm5t7ePT6AqmWLePAAQPS8NE8/rp96woheoV0ZY1pmkRdgrErMgIfBId3Ehr2MMk9EwqcVPtjRkCoZDXerzNE8MILL2RlZW3evHnLli1oCMvWI0eO5BJv7rue/KiW7H722WdT8OGHH7anKd1IDxRxsmb37t1ImZkzZ1Lnz372s3Xr1hEhk9srsobutLS0LFq0iGCGDRtGYIRXVlaGXOhrssbmglkgVGaE6W5sbGTQCgoKGL2+GXMPwLDELVtuIYz79++v/+CtiitGlY39/t6Xn2BkunfrCiF6hXRlDRul/zmNw69XOD733HNNAIH7mMk+oTDYXuOe3Piv+hWJ1WxKBeJUEadmB1dV2M+EEvdHLN61Po8FX15eboM2a9asnTt35sZA0CBr0DevvvoqlwYMGEBGdFs8iXz8+PG8+4cZM2bQcfuYg1d8li9fHqvvf0yfPt3KAmrACsLcuXOx2HSvXLnSonJwiWRDhDU1NSUlJUiHl156Cc/TTjsNTZOXl1dRUYHoOeecczDSNFHZPUD8rgbs1ta4cePs+8U4AAeAhWMCoCwOHB955JF33303PbIagBqsWhyomVJE1V5FR+6//37LjjYaVoTaGCgq9+qKhWT9jQsYOxazc0rAHPNKbVxi0FwM/tiAU5ZVrItjJ06cSFnXKcacPM2xnzFjxjDj9iEUmtX/hImC7oddP/nJT4jE7IabJtc7fGxIPY/Mwe58ho5li8hD6iG+6xa9WX75SASN/VX8YwaaDztzGiA0DZk4aEL0fdKSNSYXvJOO+C/ZVogc4djkiFNC/mO/EuLAHceJEmrzX+XAarZjp5lMRblj5x8G/v2RV46xeNf6PBY8b9kZLpgzZw7qgV5UV1eT+fLz83fs2IGysasc2JtXtIhZHIMHD54wYQIHyBfSwJIlS8zuePjhhynLnk7e9UwHQJSYLlmxYkXiXo+FUigqsi9xEiGeQ4cOJXkTZ0NDA5es+F133UUYHPBqyYN0deqpp2LxQ4R2QL7HbdmyZRxTBP1hdoM4qYHW4zqLW1ZWFn30zn08+OCDLS0tKK3ERsEEnHUH/YHFH7AFYx8VEaG/BgZ2w4YN/vAobrEBwiguck6pmQOGhTiZX7M7zjrrLIKcMmUKx6NHj2YMaZ2qEqcViIRh5CotWtiJvduzZw8O3oRlCHTHlm3JqEHc6tXvzfELmh74aykpYMa9aIQQAZGWrEFDIDK8k474ZQ0H/i8Cu1ImcfzfFza7X5EYfr3CJXMz3CV/i0C1nJoYwt8fQOC4/TECT2vefPNNBE1TUxNagde9e/cWFhZmZ2fb1YULF2J3P/+5+OKL169fX1lZSdLl2IwPPPAA+qOmpqaoqOiMM87AMnnyZCbCqrUHP3Dfffft2rWL4frwww/dF5MXL15s+dUL7gBYSAAoBuKxiR42bBj10xDxEP+YMWMwDhgw4IknnqBp7BhRUaa0sD///PPESXMvvfQSp+2N9etHOiekTz/91E5POukkOkIKv+qqq8xCDcTjJBc10Jdrr72WgryJRxm8/vrrXDrttNM2b96MBKSPvOk/5ZRTMNKp999/n4Glm/fff397df36LV++nAqJOTFgG/OHHnoI+7HHHoudaE1yAbW9/fbbdN8pD0aDMWH6TBjhwOwQ4e7du5kC86EVNxe33HILFsYNOYg6JLC7774bC7KGsaIqiscKtc8gMeNA/IwJFkaAoaBF+1QRXHg2mKZZM0vZ2J3PiBWPHlT22xFxmqMH/kr/eu/eWU8mveGFEN0mLVnTiVxw0iROu4CTNXFCxD13cQcO15CJFX9tnJqze5xj+LVRYoXBEiVZQ+4kC7qn7lwizefk5NjVL774goz+hz/8geOLLrooLy+PJEoKxIGkfsEFF2CfOnUqWdO+CnP66adjueGGG7Zt21ZcXExWJmFjIQv6C27cuHHgwIHYP/jgA1J70tFj6ycqYkN44Ul6LikpQVvgX1dXN2rUKIyXX345t0FBQQHdwUijGIFq/c0hpMxOSkOILFiwwE7RHKR8RJvTbaZ7LrnkEo65A9euXYsDHWFMgIbs+8s//elPubRjxw4uWXiDBg3Kysqi0Z07d9IupW688Ubsl156aaqA6Rr2e++9F/uZZ55JqIRBDcOHD8fypz/9CX2Js4vt448/piqTia4518dHHnkE+8iRI91cUAMW+/COeqjNnugga+gjosSewTz++OMUd5EjXPr374+dqUch4cyxhW1jdccdd2BBpFIJN0wGZWhuM2Jm5P894pjdq1fk3zMhTnaE/bdj9Hfzzv4eSyyzxk2IPk5YsoZLwEGcdgEna5AaCA4zskvihl5xB2Y38LeG/GIF/CrHLvmxypNWGCwRkzW1tbX2zhs4INfOmzePS0cccQS5kKv2pGHu3LlbtmzZtGkTaXL79u0k9eeeew77rbfeitwhVWMhiWKZOHGiPcxwrcQVRH9cd9112N99912URNJd3oKh9bfeegvPM844w2QNmQmFgQ7AePPNN1MVudYklMkXZEFic8cffzyXCJJKrEIsOGRnZ/OKMxagZtTGkiVL6LtZaBeRZE87uLVefvlljMia1atXUy19tEcgEyZMoB56bXBsn50NGDCA8ayoqEgMGIEF9m/J3HTTTcRgYQ8dOhTLK6+8QvCER22cwnvvvYfOePDBBzlGW8T10ZYG0bq5QFxi+dnPfoaIpEXUz+23344FgWXqkGP48ssv4yI3XfWXv/wFFThixAiOn376aew2VnQBCwqJAW9pacmsO7+xsZGp3H3mQHpKx1e9N3/bzZf7lce6qZMWLVr0ySefLA6OTz/9FEm6dNgxG0cOzh3z3YwbNyH6OGnJGr8u8WNKAkHDcaKPe3bixArYLkxBKxunQlxt7VrJ9wmUUzlW3F/KVW66imrNHgamDCIpa/bv30/Os19CTZ48mbRNB+0t+zPPPJOVlWVZmeHlFQt28hwps6ioCIul5D/+8Y+kVSwkb07hs88+8xckJV999dXY58+fT3OpPodKKmuQQQRvn3bdeOON5FqCJAEjR95//32M6JW4OGnuhz/8IZeIk5RvguP0008nSXN1165dJDksQMBUhQ+3FnrFxFD//v13797NOHAVtYEFrbBhwwaMyD77wgoqjdrsR/JUSOuzZ8+2skglCiYGjNyhTvsaLyOG+ECgUNDG8MUXX8TTquIU0JpUZU93aC6ujytWrMBOWTcXSCUsdJNq0Tq5ubn2sRSTy6gykhzDypUr4yIfNmwY9kceeYTemcR54YUX3FhxV2BByzILGXfno4mRNXtGDKSbDCB9/+ijjxbNfmnTH8ebrFl/zx/RNNyuy5cvZ0jTZ9myZahkBPeKMwZtHjn432O+x22cWeMmRB8nLVnDBsqOFidBAOHixAcHflnjFxlO34BTP1aniRjDaRfgwBWBdo2T7JmQVWKerubwMGUQSVnDwbXXXov9iCOOYFO2BHzllVdiGTt2LLnNLPS6tLT0F7/4BXZkDSmTbEE2teRN5iOPYiEvHn300VhIxv6Ca9euPeqoo7CTquvr6w9V1lCzNUTTTD3JnkpQCV988QVGoFp/c++9957ZSczY3XeQOaVONAdJ3RwQNBUVFYSNpCDtoYf+/Oc/Y//rX/9KE9RmHzmhFdBtFKRye35DN3n3T20EY891rr/+euznn38+Rqo1reACtk+7eONusga5Q4W4cdU8qZbwiISx5RS452nu6aef5pjRIyX7+2hPcdw3aZgLlIdZqBMHnE3WjBgxgnZRgSeccAKnTI2Ng0VOOrcnVQwaCsnG+aWXXnJjZY98qIrjxsbG/2TOd2BN1jC/BSOPYZSAWV63bh0iZtGiRR+99s9tN1++9c+32uMrpgO4MdKEJhCsTNbGkYN3nfXdwrO/x0TYE0ovLCFEeqQla8A0h1Mh7JicmtQwOJ00aZJz4NTUhnk6uxMogL8TIqZXzC3xkYyrLe4Sx2ClqNY9EwqJ6Mma1tZWNt/nnnvOvo7av39/xATvaMmOJE42ZXNGx5AG2JTZrN1Xhu2hDm7UaW/uSajkPCzkabthgPRJkiav8M7VEiqkKWtuu+02GsXS3NzMJaK1H2yTmP/xj3+YdCAl25dFwC9ryPfEQ4JnBslw5kDAnE6bNm3gwIHz589H1vz617/G/sILL1A/wdg/eWzqgQQJHPzoRz/CSKd460+LlLInJYA4oBUaNbHiAmbAW1pa0Bb2fRcTgowY+sPG8JVXXqEgFkayvaLYh4B0kHVkSnHIkCELFiygKhzcV4Zpxc2FfQmGgcIByUJtTtZUV1fTTRNkMHXqVKolcqZj0KBBWFhE9qzIxtmCsbEyHUbABEPrmSVruPMZiuJRgxhnRqld4hQUbN++HSnDZrJ69epNmzYxFIwePbVPCdPERn7z5s17zvpuwdnfKz7n+9TPvSRZI0RQpCtrwASKwykVsEtsEAiLxKvslVhMc/if3IDzByoxo2VEOwaTMnFXDbtkbXHgrzkMoiRrEjnmmGNI3uzvu3fvtv2dfdk/2sbJJ59sX0olWZJKcSNJ2DdIeENP2mBDJ2eQ4y+99NJYif9x5ZVXWr6koXRkDSmWzIE+wBM7iXbFihX2yZEfe8IBfllDDdRD62gLJIU5EDCn9pzDgUji7Tt2gnnnnXewoB4oQpfpIzckss+UTRwMGu/XGT0ajQuYrIayoUKTNfbwA91ArrUxfO211+gvFoJsrysma6gKFYUSdV/9MQYMGGA/YjdZY3Nh+gORxFxgYXCsX8gaF/njjz/eXr4jCCaGEVHrZA3BuLFyOixDZQ19Lx09iHHmXkLJ8WrKg3sDOGCCGhsbmSMmKH1okQpZSkXnfK/knO+XjpWsESJgApA1nYCwYMvzTqJLBGSNS5aOY4899qKLLnrqqad488qbS96ss8WTxurq6sivJLmXXnrpvPPOGzly5FlnnfXggw+yU7s37pZKXUpG1lA/KZlUTcENGzb85S9/sYJjxox5/vnnyc0kYDy7/bTGHmmQYomQ5EENpCjyN80tXbqU/G3NXXLJJQsXLnQPY0zW2I+0qYHcT4U0gdEc6DKnhHTffffZMx60+AcffEANJD/acrKGInQQUEKMFcKaYOxRB1xwwQX/+te/kHS0yCDgY/rABcwsWO9MJRAwwXNKKzaGBEk3cXYClMVFVbSLxlqwYMH48eOtj9dee+369evd78XcXDhZQz1kVr+s8Uf+yiuvnH/++e0NxH5gNWXKFIzcADt27CB4Cxsh6MbKvkuU0bLGli03DLcNNDc30xGGmjHhbscHI87cgelDVdxO3D8IGvv6jmSNEMESrubwf5wUYeL2x0yUNSRLshfpcNOmTbwpZ6tl5+WAlElKQxzk5eWRC9n6gVxLhtsYAx/SoeVXV4NLpVRCWRQPWZAUiIXjVAVpCMtBZQ2RoA8oQuI3WUPw1EZDVIUDbiQJkhMxdBInRq5iJCQLkvCojRow0gR9R9aQ20xMUHDnzp0ECRTEGTsV2hd7nQDCk6oswpycHKJlEJw0pGaGglKoBH/AzAKvJFEs2GkI8WFfImYuLDxqxtnqx0Ll+DAIdC1VH5POBTXQqUOKHDeObawYFl4Tx4oKM1rW2LLlNuOVXsSerbTavZR4N3YbauOmZU5N00jWCBE44cqaHvheS18g6f7oXevzECoKgCxFKiURrlu3zr4ba5CusJO66BeChl2evZ4UaEmRVGrZmoSKG872exzeuJPykUEU5BIbN8XJeeTpzgu61JhK1tA0aRhNQGZFHzQ2NprMIulSFcVxcCqBbNFJc7wSGA68uiCp0Hpn3w/lAAVgn0rgQxGEAnGiFegd3UEKIHG4VBL7Ty6ZejI93acso0ej9gqUogkTQ5RCNyQGbE1j5yqenNK6NU1ZwsDCqwVsk2LPXQ5pLiwGJFHXI2d48SdstI5VkjhWODAXGZSeO1m23GzAKXBgxkCQrBEibKL/CVEPwN6Xan/s+7Bro1TIc6Q9kqJJGQ5IVKQ90hjpiqt0yh7FQ0tLCymQfIa2IKECB2RioAgpn8TJTk3OQ3mQKakBBYB4gs4LkrNpKOkWT5zYSZxUTqa3bO3qLC0txUJxy6yWkzqPsz21lJVRhFd/kKaTGA1a4QCjPdugCPkbKGgfweBplyhu8440oQYCM3+GEU0ADKb1DtFGKRwQE3EBW++wYOcqPpzSihtD+oKS4NUCxs4IY7HmUg1p4lxQA2EcUuRUwjhQMG5CLWAGClBOWBh2b8L6PIRKx3t42TLLTIdkjRDhIVkTAL2yPwaFJVRiJs+RrshqQFrlrTkZy1KgfacVT+ePhUyJA3oCZwqS+eg7UI8VIcmxX1MDnhyjirpSkPxqDSXCqKLA8CH7Ui05FWcgeEu6XMLBBr/zOLHQHJcowqsLktpcE8ABRpMODIjVQHFrCIjBmsaHsjRNoxSxRvFHBDCelKWGTgImWtc77FzFh1Pid2NoFl4tYOxcpTbXXBfnghqwHGrkcZVwTCkLGAeM+CT99LDPQpcZCkZJskaIKCFZEwC9sj8GiFMAZCaSHHBAL0hd5C0u0R1/urIcbAkVZxKb8wfLtZQCy3x2SqmDFrQHQrh5LXXEFceTat3TIyrn1Bri1BU/aHNcMqws9cTq+18TZqQhTilLDdRDWezmyaVYy14wtJjYKNAoPq7+pAG7gnbJnHk1C6+pLMCxa64rc0GRbkSeWIn5+6fDRj4jIFp6J1kjRMSQrAmAXtkfg8VSGrC9gh2DZTvPqSNWxPzB/A0rFYcrBfh4xRIKmlsqXHE78BsdZnRgwd9r7GBxmj/EXeXUKx+rAYur3GEWg9PEIs7Hjh1mBO/8UHCl4ppz2NXOsdYNTinlVdQxcjtwOIvfJ4MgbMkaIaKHZE0A9Mr+2Hfodj7r4USYfnPtqfsQK+lGkXQIsK0ejrznkawRIpJI1gTAYS5rhMhEJGuEiCSSNQEgWSNExiFZI0QkkawJAMkaITIOyRohIolkTQBI1giRcUjWCBFJJGsCQLJGiIxDskaISCJZEwCSNUJkHJI1QkQSyZoAkKwRIuOQrBEikkjWBEDi/sgmFe1/80OIjIblySKVrBEiekjWBIBf1lTF/vNC/z9ILyIAs2l45yLDYSpZpCxVFqxkjRBRQrImANgNPVlz/hC2SP1F6a909KCSUYNKRh9bctbg9r9R7ZY4H/1l9t/5QyRrhIgMkjUB4GSN/d/XvP+rjf0nzPUiw7H/47pdsP52hCWhqh1ba2pqNLmZjs2s/V/lLFvJGiEig2RNALAbtrS0kO3YIouKikpLS8vLyytE5sM8kvNycnKKx59hSShnzedMcVlZmechMhYml6XKbLJsWbzNzc2SNUJEAMmaAPj222/b2toaGhp4z8deCd7GKTIc5EthYWF2dnbRuGGWhLYtW5yXlyflGg1stbJsERYsYRayt6RDQ7JGiLCRrAkA+wZia2trY2Oje8StzykiAG/iSXs7d+50siZ7+RLe3O/du1fzm9HYCgWOWbYtLS0IC8kaISKAZE0wuN9W8J4PfSOiwb59+1A2OTk5hZcNtSS0a9XyyspKS4Sek8hkWLDud4veYg4TyRohwkayJkjYGUWUIOGRcvLy8tzTmpw1nyN0SIeWCEVk8NZwyEjWCBE2kjVCpIRk09DQkJ+f72RN7tpVtbW1PfM9DBE9JGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSIlkjgkWyRoiwkawRIiWSNSJYJGuECBvJGiFSkihrctZ8Llkjuo1kjRBhI1kjRHIQLvv37yfl5OXlOVmz54uVNTU1ra2t33zzjecnRJeRrBEibCRrhEgCmoZM89W9E1z68f+Vjxva1tSoBzbiUJGsESJsJGuEiMc0TXNzc/XGtS79+P8qX3uWq/hI2YhDQrJGiLCRrBEiHvv4qaGhoaSkpODWK1wGsr/iS35a9O/c+vp6fcNGHCqSNUKEjWSNEPGYrEG4FBQUbFowz2Ug+9v+5IOkIn1xWHQDyRohwkayRoh4ECukmYaGhqKiovXr1++c9P9cEir81anrVn2O3CEVIX0ka8QhIVkjRNhI1giRhG+++aa5ubmqqio7O3vV3NkuCWU9OnXLli3l5eX79u3Tj6HEoSJZI0TYSNYIkYRvv/22ra2tvr4+Pz9/zZo1OyZeQgYq/NWpq5cvy83N1SdQontI1ggRNpI1QiTHHthUVlZmZ2d/8earZKBNj91jj2qampqUh0Q3kKwRImwka4RIjntgU1BQkJWVtfvuCRvWfJGXl6dHNaLbSNYIETaSNUKkxB7YfPXVV7m5udu3b8/JyamsrNy3b5+SkOgekjVChI1kTZDwDl5ECWRNW1tbQ0NDVVVVaWkpmoYMZP9zguchIoG3gMNHskaIsJGsCQZ2RlJd1XU/Lxs9SH/6018G/bFsTad6izlMJGuECBvJmgAwTcObeLZIdqj6+vq6GByICFBbW1tTU7N3715eOfasIpPxr1CWrXsC5y3p0JCsESJsJGsCgN3QPqpgf6yoqCgvL+dVRAnm1PDORSSwOWXZsnh75mvgkjVChI1kTQDwPq+lpYW38uyPRUVFpaWlyn9C9HFYpCxVFizLlsXLEmYhe0s6NCRrhAgbyZoAYDfct2/fV199xf5YUlJSVVVVW1urD6GE6LOwPFmkLFUWLMuWxdsz/2y0ZI0QYSNZEwB+WcO7QDbN5ubmViFEH4ZFylJlwUrWCBElJGsCwC9reP/X1NS0f/9+jN8KIfokLE8WKUuVBStZI0SUkKwJAL+ssf2RTYqt07sshOhjsDxZpHHLVrJGiAggWRMAibKmB/ZHIUQ69MqylawRImwkawJAskaIjEOyRohIIlkTAJI1QmQckjVCRBLJmgCQrBEi45CsESKSSNYEgGSNEBmHZI0QkUSyJgAka4TIOCRrhIgkkjUBIFkjRMYhWSNEJJGsCQDJGiEyDskaISKJZE0ASNYcJsT+fdr/4Vm7zGOPPdavX79zzz23G2VF4EjWCBFJJGsC4HCTNStXriQ9J4VLkczZdGrRokVjx461bg4ePHjChAkrVqxoVzdd6y9uM2bMoCyVkMO6WKobxCJqxzsXKZCsESKSSNYEwGEla+ja8uXLLbsnsmzZMhwillPpzuuvv+71sCNz5szpSn9xwG369OkUOfvss5ubm/fv3x/4KFHh1KlTaeLUU0+tqqoKvP6IwYxI1ggRPSRrAqBX9sdegUxJPl68eHEsp/crOkBxcXFpaSl9b2hoaG1tDbb79pCjtz67odE9e/ZYfx977LGNGzfm5uYi7K688sqrrroqPz/f+tt5bFxta2t76KGHqGT06NG1tbUom4OWOij+kQFS41133YXllFNOYTrCUE5RoleWrWSNEGEjWRMAvbI/9gr0i3y8aNGi9iTfr9/mzZu3bNmydevWbdu27dy5s6CgoLq6mu6zRweSUKmEFh999FHaGjt2LMc9n6dpdOnSpQQwYsSI7du30+VNmzbxmp2djb6pqKhAyR1UQFBJS0vL/fffb/WUlZV1pVQnxI0MAw4IrPr6+srKyvLy8r179wY4EZGEAez5ZcuMSNYIESrByBpWJturgze13oUugPPnn39ux7zvnDRpkh0HCzWDdxI0vbI/9gpsvk1NTe+8845N9Lp16zZu3IiyCUPWWOYmVdtDjrPPPhtl0MN5mrYQH6tXryaAI444Yu7cuRs2bEDToOSQNbt37y4pKUFJtLW1dR6V3SH33nsv9QwfPrywsLCuru6gpVKRdGSojampqqrKz89Hb5WWlpIsu93E4UCvLFvJGiHCJgBZY7/veOONN7xzKo09GPdOOoVSOLOwvfPQOO644w5JbB0SvbI/9gpsvg0NDf/617+YNSDB79mzhzxaVFREgienskG7D2VWrlw5btw48/zJT36CJkhMsQwXQnZsjIkTJ+bk5Nj3kbl/qISsbMUd55xzDjHYsauNA/yx2BeWwRw4pdEjjzyS2cdozqR8GsKIA/YZM2YQg11KhFJ0p6am5swzz7Q6x48fv2zZMnQJFBcX0+XGxkZ77gI0R0/Nk7677xTbHTJt2jTsZ5xxBiNWW1trmuOg8eBjI2k+BjF4RwdA3BCJKSei9eutuCY4dk1w1T7J4nXRokUWPD4zZ87kkvlElV5ZtpI1QoRNurLGNE2iLsHYFRmBD3uodxIahEc87plQ4PTK/tgrxMmaRD777DN86D4J3jP5mDBhApcsX/K6ceNGf6oGTu3bIcgX9ERTU5PZHWPGjGlubrZjqwpoEVWEBRmBkVNzeOaZZ+yAdlEeXEpsETr5gi1Gwqiurv7yyy+HDh3qFejX7+qrr6aquro6lISTccgF77KPOXPmEA/QF5M1w4YNy8vLQ9ZQsPN4jKTfVv7000+9owMwMkyNfc41cuRIp7eSNoEFO2GDfZJFo3bJYd+G9gYiitA7yRohoke6sobtz/+cxuHXKxzzZtoEELiPmewdtsHajnty47/qVyRWsykViFNFnJodXFVhPxPqlf2xV2DzJXfOnz/fRjiRpUuXtrW1ue/YkmV37txZVla2YMGCk046Ccurr75KJYwPDB48GAt2rpaXl+/atcuyMpCk6+vr9+7dW1BQMHnyZCyoipycHPvWiPnYZ1JkbgYcfyxLlizByKk5HHvssYsWLUKUoD9QFcgIa/Hiiy9et24dUa1du/ZXv/oVlj/84Q9UhYbw+nkALHQHCcLNgw545JFHjjjiiFjd7cyePZs6rTsmPgYMGPDiiy9WVFRQ+cyZMznFuGPHDtziZE1NTQ36rPN4qNaN5FVXXcXV0tJSRsC+PUNIbmQ45sZjxB544AEso0aNsu/W7N692zRNYhPERiUMl32SBWg1/JmvE088kdOzzz6bvhODNxaRg671/LJlWiVrhAiVtGSNyQXvpCP+SyZQkCMcs4Y5dkrIf+xXQhy44zhRQm3+qxxYzXbsNJOpKHfs/MOgV/bHXoHN1y9rsrKyyIK5ubn5+fmFhYWkcwQE3b/++uu5euONN3IJh+zsbFK7ffEWqYED+RJ9w+mgQYM2bNiAD5DCmeWHH34Y+8iRI8nBJSUl2P/4xz9iOe2007Zs2YKyoS1OgWRAVkYcIDtI5Fg++OADwuPUHP72t79ZbFSFcdasWRhPOOEEf1Q0eswxx2DnmN7FKRtOrcvUgAP9XbZsGZojVn077777LroKHxMor732mr/yJ598EuM111zT2NhItE7W0E1kR+fxbN++nd7dcccdHF944YXYuYoRTwaB061bt95www1cZWQoy1gh4O677z4s9rSGsK34L3/5S2uC4lRCcRrFzvighOxzq4EDB+LDJcTlnDlzsIwePdp9vuYNR7SQrBEikqQla9AQiAzvpCN+WcOB/4vArhTrmUv+7wub3a9IDL9e4ZK5Ge6Sv0WgWk5pgmP8/QEEzmEra1avXo0ocV8ZRkCQTcmUZ511ljkkZePGjU1NTQ8++CDHt99+u/22iCRNxqUSm7gzzjgjLy+voKAAo8man/3sZxQ0n/Za+vWzBxKkXuSUffflvffeq6mp4dQc0A3ke1K1P+Wn4uWXX0ZGJE4cltbWVuRacXExTRMtYcybN2/IkCGUQqXRF/e5WFKQKbSOrrrnnns4NVnDQHUez4svvshIoi04Ri0xyMAoIU1QHkAk3NVcRdYw/vZFH2ti+PDhdJmhsIl4/vnnKYu/wfFtt92G/aKLLmKs7FO/a6+91irn1fTWiBEjGGH6LlkTIJI1QoRNWrKmE7ngpEmcdgGKmC6JEyLuuYs7cLiGLOf5a+PUnE3BOPzaKLHCYOmV/bFXiJM1a9euJU0iHcj3e/bsKSoqovtkU3t2kop33nmHfGkPCcimqASyKdOHiMnNzbWvDA8dOpRjUjWvN998M5bTTz8dN1rhtb2Wfv0qKyvtKQgpHBmE5V//+hcBcGoOL730EoFZVLRoKT8V06ZNozY6GJfFKfv666+fcsop9LSsrIw4CYDjhQsXWkGTLHacCkKinrvvvptjZA1Ki6rsNBVTp04tLy9HW3D8zDPPmD6zp2L0iNfdu3ffdNNNXGWsGBYECn20Ou1byVhGjhxpxRErKCEGE0+qsk+vLrjgAsYcZcnxjTfeiDaiCaqdPXs2FrQRNUT4ZpasESKShCVruAQcxGkXcLIGqYHgMCNrGzf0ijswu4G/NeQXK+BXOXbJj1WetMJgOWxlDckyLy+PnG1f+CCt2hdirrjiCq7OnDkT3YNqwQ0dsH79+nXr1m3ZsoWMi//TTz+Nz1FHHbVq1SryNOIAPYTdnuLYIw3SKpXbowUsZGXSMOmZU1i6dCnB0CLFjz76aCzz5s2rqqpCDJnDK6+8QvonHtIGlVuLN9xwQ9Ko8KSqxM9cOOVGouCll15KPcRJ/YiDFStWxBrpR0E0ysCBA4844oiPPvooVZfpi/0TwHTEJODf/vY3TpPGw4GJmN/85jf4nHfeeRzTtbq6OoLklU4xMrfeeqtVSG1IK+xOOZkotE+pWD4mpAieaWLKfvSjH2F/4IEHsFslKCTzoVr7nhDaiNOmpibJmgCRrBEibNKSNX5d4seUBIKG40QfTjFy4MQKmEChoJWNUyGuNoqAGcGpHL++MVzlpquo1uxhcLjJGvdLKHIn/cVCl5tjcICAsP9d4Tvf+c7DDz9M+t++ffvHH388efLkoUOHrl69GmlCviS5mhYZMmTIwoULSZ8kVPvGK5CYcSCRI3TsCyJkWQQEqXr37t1MLpaLLrqIqsjlU6ZM4RRVsWnTJr+sIT2Txe1DIqLiHkBFYb/xxhs/+OADokJRzZgx4/jjj3/nnXcoRXaJkzUcM5ULFiywClE22dnZtLhkyRL7EOqyyy4jHsJ44oknOP3BD37w5JNPomOonFEizmuvvZZjIvfLGivCa6p4EI6MLQEzbjjA73//eyppaWlpbW3ltbGxkZ7ayAwfPpxRInhwsgblRBPEad9xvvDCC5FKDAXjb8/SaJr1gka85ZZbOOUVbcQoMaFvvfUWFgacGaFIVDOuZI0QkSQtWcOCZPuLkyCAcHHigwO/rPGLDKdvwKkfq9NEjOG0C3DgigCVW0NWrRnBKjFPV3N4HLayxp5wtLW1YafLQNKtra1FkdjPhhN56qmnyMGk5Pz8/Lffftv/wyIYMGCA/cdJJGYcGEx0yZ133omF5I3yQA/l5OQkFoTHH3+cePyy5s0339y7d69JAYsKC2LLrvq54oorKGgpPE7W0Du6/OKLL3quPtAfS5cuRY5wv6EPED3eBR/9+/f/8MMP0TTV1dVOczhZw32bNJ7LL7+cXuBD5fbvysSxePFiAnYjQ/0EmShraCLpWGFBu2zbto2w7WnNbbfdxtgymyibefPmYZGsCQMGU7JGiFBJS9aAaQ6nQkxPmNQwOJ00aZJz4NTUhnk6uxMogL8TIqZXzC3xkYyrLe4Sx2ClqNY9EwqJXtkfewU2X7+s4f09p04K8Lp//372aOxbt259+eWXzzvvPPM85phjkA4rV64kd5I4ERmkzOzs7Pfff//Xv/71yJEjzzrrrGuuuWbdunU240llDUXI39w5mzdvJiv/4he/iNXd/rwHtWRfm6UIbmYno9MQmgZp4qKixfHjx6M2cOD1ggsuIIsTVV1dXeJXhjnFiDZCMBEYzlZw0KBBRLVs2TL7LRgCIjc3d8uWLWgy+3wHTjjhhClTpmzatMl12X1lGGVGMHSEIknjoQt0nIKoE3zmz5+PPVarx0cffcRQuJHB2b5mFPchFE0wLOgqdJI1gXBkIlgpVEvkRGKf8d1+++1USA3E6WRNaWmpZE2wSNYIETbpyhpgWbIJOkxMGHaJPRRhkXgVwYHFNIf/yQ04f6ASM5qEsmMwKRN31bBL1hYH/prDoFf2x16BzZf0SdLdHoMD+5qtd/nAUJBlLR8jI+ybqkAGZbJMQCCGqqqq8CHrb9y4kfyKiEE64EAaXr9+PcIFS3V1NW4YEUA4U5bhRRDgY/+JATFwiRqAhkjMCAjcOMaO2qAte5jUeVT2IQ6e/kc1QHeam5tpFE9KUdZ+IL17925OLQYrzlBQZyddrq+vx80Cc0WoJ1URtBTdZ0zwpxW6TPBcolPoFXqKD842MvaoiVHFyGBiZPTwcU1QnAOK48xVLAxdXl4e9ZuFV8aHGgiVEeYq9VCbZE2wMJiSNUKESgCyphPsnbd3El16ZX/sFayn5EsSKnAQ11n71IYUTuolVZNNgawJCALStgkItAI+paWl5E5ytqVtjq0IpziTkmtra8nuZFnasoc3JF1SOPWQmFEVRlZWFvWTodE0VI4DCZsi1GBihQg7j4qOkNGTfl+YgpbpcaYha84OTEuR+4mTpjvvsik5AkNA4Em/oJMiREsRukwRrqL/gF7jg4P9u3xIHGpjZKiKiUCCIE24ipG+E5JrwjSZDbVpGtM9+JuysUqogVBNMFEPATNT0b6Ze3jZStYIETbhag7/x0kRplf2x16BfrW0tJDFSavAQdwHN+gAvw+pkZxqeoVjy7VkSrQCPsgFfMigOADJlRRuICOqq6vJ6+z4DCluJGAyPWOLkWN8UACmBqgcf0vtVE61JGyKUAMWEkYXo0qcNQpSnEtUTvqnCJoANYPCQCjs3r2b2ogKBxrqpPLEwOgFGqLzIq2trY2NjXQf+cKlmOxp7yyRIDgIiXGgOK+Mkn2VmCaQOxhNo/iboH6KI2gYZxteShE8VblKqIF2KYuFeqiNahOHJRrQLzrbw8tWskaIsAlX1vTA91r6Ar2yP/YKlubpIOkQODDR4F2OYT4kSDZrcieJnHf/5EjSJ2mS7L4/9t8z4UPKNB+uApmVYzI64ElKtlRNaqctdADHlEUScQkfkwhU7q8ZOKBaiuBmj2q6ElViRwwriAP6gFJIivwYxbF//o7aCBIHSFU5o5QYGKVM26WKB38i55XiNM3g4EB/8aTvjAZ2Xt3IEIC/Ca5SPxbXBJX7hwuHVMPLJSyU6mRYIkCvLFvGk4GVrBEiPKL/CVEP0Cv7Y29haZ7kB6lyHkZGAAcyJaNBFueVHOkvYj6Wd50Px+RaS/l44mA+nPLKsZXiEj5WCvw1AwecmsWKdD2qpFid1iIZCE1gOsDKWhPQeeXgAvP3hdNOioA1jb29qwdGiRqwx42MObc34Os7rzi4JsA1wSW7yqmrhFd/JVi8UYgc9JQxkawRImJI1gRAr+yPvUh7/jyAZ0oGVxkHP4lFUvmkwpUCr0AMs/ivGmbxg9Erc4BUnn7MB2cnAkwWxBU0Hz+xcp0FxrHneoBEB/CuHcCMfvzOhlmAY6/YAZyDHTicv8MskYRxyAhZ0/4jiAR66yn4GzG8EyH6JJI1AdAr+6PoFbxsH8Mzicwkg2RN3G857ZeeYf/AMxFCpV3JGtHHkawJAMkaITKOzJU10Cu/xpCsERmBZE0ASNYIkXFktKzBgt07OfD8xvB/PmX/HhiviZeAU7MDNXjWGK4IWOumaQz3T6cK0QeRrAkAyRohMo6MljX+pzX+f30UsDv5YurEFefYXTJNY8dWg1M2/hr8lZuy0dMa0ceRrAkAyRohMo7MlTUmNZzx3Bh2DHaVajlGoPgvoUjsEsQJFHSMeTofswOnpnISSwnRB5GsCQDJGiEyjgySNYn4tUXcKTgLssYviUyXoHtMu3jWGIkWczYka0QGIVkTAJI1QmQcGSRrnDQxYeF/AOMXH36sSFJZgy7BaG5x4ICbfXQFVpYDyRqRQUjWBEDi/sgmpR8AC9FnYXmySDNO1oB9xuS++9K51Ej1tMZkjWftSOIl15xkjcgIJGsCwC9rqmL/l7L7N1uFEH0QlieLlKXKgs0sWQOIDIyoEztFuziVA37xwaXE79ZwYNrI1QBOzVAVpcwIVpur39UsRJ9FsiYA/idrzh/CFqm/iP2VHviLs+svIn/nD8ksWQP2OZEdm1hxagMd43SJudklEyiuKtxcDf5LVptfM3EqWSMyCMmaAHCypri4uKSkhPd/tbW19j8HiYyGSWQqq9etrF61ZO/qpbVVlZrWaGAzy1JlwbJsM07W2OMWJ1/s1PA/nsHBHu0YcfX4L/nFSpzdTv2XXLtC9EEkawKA3bClpaWmpoYtsqioqLS0tLy8vEJkOExiWVkZOa/0N2daBira+KUmNzLYf2nOgmXZsnibm5v7pqxJB5M13okQhweSNQHw7bfftrW1NTQ08J6PvVJpLxowj2ianJyc4vFnWAbKWfM5WRCt43mIDMdWK8sWYcESZiF7Szo0JGuECBvJmgCwbyC2trY2Nja6R9z6tCLTqa2trayszM3NdbJmzxcryYK8s9fkZjq2QoFjlm1LSwvCQrJGiAggWRMM7rcVvOdD34gIsG/fPhRMXl5e0bhhTtbwzr6pqYks6DmJDIcF63636C3mMOlhWSPEYYhkTZCwM4rIQLZraGjIz893siZ37SqEDrnQsqCIEt4aDhnJGiHCRrJGiOSQaRJlTW1tbc98CUNEEskaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEiOZI0IHMkaIcJGskaI5EjWiMCRrBEibCRrhEhOoqzJWfO5ZI1IB8kaIcJGskaIJCBc9u/fT77Jy8tzsmbPFytrampaW1u/+eYbz0+IQ0GyRoiwkawRIh40DWnmq3snuNzj/ysfN7StqVEPbEQ3kKwRImwka4TogGma5ubm6o1rXe7x/1W+9ixX8ZGyEYeKZI0QYSNZI0QH7OOnhoaGkpKSgluvcOnH/oov+WnRv3Pr6+v1DRvRDSRrhAgbyRohOmCyBuFSUFCwacE8l37sb/uTD5KH9MVh0T0ka4QIG8kaITqAWCHHNDQ0FBUVrV+/fuek/+cyUOGvTl236nPkDnkI6SNZIw4VyRohwkayRoh4vvnmm+bm5qqqquzs7FVzZ7sMlPXo1C1btpSXl+/bt08/hhLdQLJGiLCRrAkM3ruLaIBkaW1traury8vL++KLL7ZPvKR07PcLfnXqqmWf5eTkuN94e94iw/EWcI8gWSNE2EjWBAA7I0mu6rqfl40epD/96S+D/li2plC9xRwykjVChI1kTbqYpuHtO1sk21N9fT3v8oEDkbkwg9XV1SUlJdnZ2StXrsy6/ZoVSxZv3bq1qKgIu+Y3o4ktUG8GWbbu2Zu3pMNEskaIsJGsSRd2w7a2toaGBvbHioqK8vJyXkUEKCsrKyws3LFjx9q1a1E2a9asQeIUFBSUlpZ6HiKTYakCy5bF22O/a5OsESJsJGvShfd5LS0tNTU17I+8lSfnSdlEA+axpKQkLy8PZbNly5bt27eTgYqLizW/EYBJZKmyYFm2LF6WMAvZW9JhIlkjRNhI1qQLu+G+ffu++uor9keyYFVVVW1trT6kiAZMZXV1tWVBXjnG4l0TGQvLk3lkqbJgWbYs3h77aZtkjRBhI1mTLn5ZQ+Zj02xubm4VUYHZZH6bmpp41cxGBqaSpcqClawRImJI1qSLX9bw/o/8t3//fozfiqjAbBreuchwmEoWKUuVBStZI0TEkKxJF3ZDJ2tsf2SHYuv0Lgsh+hgsTxZp3LKVrBEiGkjWpEuirOmZ/VEI0W16a9lK1ggRNpI16SJZI0TGIVkjRFSRrEkXyRohMg7JGiGiimRNukjWCJFxSNYIEVUka9JFskaIjEOyRoioIlmTLpI1QmQckjVCRBXJmnSRrBEi45CsESKqSNaki2SNEBmHZI0QUUWyJl0ka/oIsX8/9tD+FcRuFBHRQLJGiKgiWZMukjWpMNEA3rkP70JAkoJ6ZsyY0a9fv0cffZTB70q1+KxYsYIikKoIRi4BKScOK5K0lMgImEHJGiEiiWRNuvTW/tjHIeVXVla+/vrrpgA8awxO2cSTXuoG1EA906dPR6A8/PDDra2tB63WiixbtsxkTUtLC/HEFeF0+fLl5pCUtra2bsdPKaC4cdBKDtVfHBSGUbJGiEgiWZMuvbU/9mXIu/v373/33XfPPvtsdIZfNNilpUuXnnLKKUn1xKFCcRTGn//8Z6TG/fffT4bgtPM6rcinn35qAqWuro5I/LOGA4EtXrzYHJJCQyahvDJdhsoptWHDhsGDB1PPa6+91vkgmP+cOXMGxFi/fn36gyYYUskaISKJZE269Nb+2GcxTcA43HrrraTtjz/+2EQDdnfpkksu4VJBQUH3lIEfijc3N993331UOGXKFKaA087rtCKLFi2iCJSXlzc0NCC2CM8cOCCw2traoqKinTt3bt68+aabbsLzxhtv3LhxY3Z2dl5ens11NxQGrVP56tWrBw0aRJ0vv/xynKiKw/xnzZplsmbVqlWd+4uuwAD2yrKVrBEibCRr0qW39sc+i2mCurq6Y445hrQ9evRoJxoYGVIyyqBdTfTr9/zzzzc1NXVDGTgoSHEqueeee6jw9ttvLy0ttTo9j2QQBj4LFiywMAoLC4nW/4zHuoCsQXghYtavXz9p0iQ8J06cuGbNGlROTk5OZWVlN4K3gLlJqqqqqBx5lCiq/Byqv+givbVsmU3JGiFCRbImXXprf+yz2IB8+eWXJhogKyvLHjCQjEnJ9uQDrrjiijg9ARwDzoadetcOYEZzoDhZYerUqVR4yy23FBcX04SpDcPcwDs/oITeffddCwOtgIKJkzUWKtolPz8fZfOnP/0JzxtuuGHTpk179uxBPFGkubnZ35DhaoC4dg1qpvWKiorc3FzkUUlJSX19Pa37Pf0FLdqu+1sAhln8Vx2ex+EKAyJZI0QkkaxJl97aH/sm5Es2aATBM888Y6IBrrzySixkYsQNubl///5HHHEE9oEDB1ZVVbmPVCzdvv7666eeeqoVPOeccxYsWECFOHDJfDheuHCh+QwYMODOO+8kMUyZMoVTZE1RURFtIR1wW7FixWWXXRarqd8pp5xi32LBzmtjY+M777xjlygeJ2sAt9bWVtz27t1LnfaBGoIMYYHCQI0hNaz4smXLqPnII48cPHiw1Q/ooQkTJhA//OEPf9i9e7d9Q3ns2LE0xE1SXV1txekgWe2zzz6zq+vXr7eYqZBOUQ914u8+MqN1/B955BGOp0+fTnEbClp/9tlncfYP1HPPPWff4GGgiKe8vPzRRx/ltOs/GYsqdL9Xli0TJFkjRKhI1qRLb+2PfRMyJWkblXD++eeTPidOnMgrZGVlMTKohMcff5zTSZMm/d///R8Hixcvdh/lMG6k3ph7B6677joUhvO56667vAsHOPHEE+1DosmTJyNBSBL4IzXsqh+rigiRPp3LGmsLeWSTi3jC054G1dfXYwQr/uSTT9oBlTc3N1MP0gQZYUaD09tuu42DMWPG0F+gRbuEXqHCTz75hONjjz02ruD1119PwFT78ccfm4Vg8LevSJ988slmdLzyyism6SBxMKmcIDl46KGHTE36u3xYQd9tZiVrhIgYkjXp0lv7Y9+EvpMvGQd7HjN37tyLL76Yg2nTppGMa2pq7As3CxcuvOiiiziYPn06ezpSgG3d9Er//v2ffvrpPXv27Nq1C8XAKcYHH3yQ1E7CtqcacMUVVyxdujQnJ2f58uVIKGsOWVNYWEhD27dvN7cpU6Z8+eWXVPXGG2/8+Mc/xvLyyy8zRzTauawx6A6qgrAttltvvbWsrAxJRDB1dXVWnB7Nnz8fuVNZWUnTSBbUCXaao1E6gqSzz8hg5MiRe/fuxY1XsyxYsKC6utp90eeEE06gR/Trt7/9rVnQgtT54Ycf2mlFRQX+9957r50yDtSPkLLenXXWWfSOvrz33nvmcPnllzNQhLFixYoLL7zQjBQnhqRdPkzorWUrWSNE2EjWpEtv7Y99ExuNJUuWkDt/+MMfrlq1aubMmRyjTtAWdnzppZdi/8tf/sIxigTRgBJCLnAK77///rZt27Yc4Nlnn8VIcZQH2d1+QvWLX/zCrmZnZ+/cuRMRM3ToUOw333xzQUEBFV5zzTWcTpw4cevWrZsPYMoAFUJV0BVZgwU7V03W3HbbbSZriAQRY8Uff/zxHTt25ObmlpaW0jSyCePAgQOREZs2bSIArhKk/VZr+PDhJSUl6JLy8vJY6X5IIixvvfWWnX788cd0x8I2i8kghsVO0W34WzxHH300I2D+//znP7GMGDGCGNxATZgwwfrOkDL+u3fv/vnPf479zjvvrKqqQpwdtvdqby1byRohwkayJl16a3/sgyAC2J3J+nfffTe5E22xIYZpjilTppx55pkczJ07d/369SZ9gPxKGrZ/JGbIkCGmV5AC5GDUAAnb3HAgwZ944okck8JRDKRzxEReXh6e9pWRm266KT8/nwpHjRoVK5SctWvXMlPoCTvthqwBRIwVf+mll4gEOUW7SAp7jjJ58uSsrCzEBLERJK/2jxoPGzaMCCsqKoqLi2Ol+7355ptY5syZwzFCkCJ79uyxX5WbAwKIjjsRRm3420daV199tfM3WYNswp+AY779Zs2aRRhIH3zoZk5OziuvvILdHjsx7JI1kjVCRAzJmnSRrHE4ETBixAhy5zPPPIM6IeVbxrXPiZA4qBaM5Nof/ehHWN599926ujp7lIKFq4gAsj65H+lAJsYOCxcuLCkpsZpffvllihcWFqIkGHOc7dnPzTff3J4wysrMLRUIGiqfN2+enXZD1pCHnC5BOiCtiKS+vp6OOEmH4DAFwyUiX7NmDXa6T3MoD4KPlW4XeVhMbXAV/YGz/Xs55kBD9PFf//qXnTI4+N9yyy0cI+OcvwmjM844g+PKykr78A7JZQOFhUrwfP7557FTvCu/hI8wkjVCRBXJmnSRrHHQ8ZaWFvIuiRNIzOR1Uj4J/qijjjLjU089ZXZ2c/t50Z/+9Kfq6mpXivxNYkZJoB5QCa+++ipGJJGV+sUvfsHp2LFjkTuMNokZKG7PgSZPnky16AD7YsrMmTPXrVtnH8Fs3bo1K4al+QBlzWuvvYZMwdjc3IyyefrppzHSX5pGTGAnwr179z788MPYhw0bRnP4FxQUxEq3y5rc3FzrpokSnFFCGM0BC6d+WcMlGzr75Zf5v/nmm1iogZoJ0r6y7QbKvqCDp9kpLlkjWSNEJJGsSRfJGgdbM5nSPTYg+6IeSOFkYkvq//d//7dx40Z7hlFSUvL6669j/PGPf8wp2K+ZUDDPPvss+gCHv/3tb/bUYdq0aTt27CABIAI4BdIzGoXRxn7xxRebm5M19gnXd77znenTp69evZoAFi9ejCgZPnw4pQgJHZDOh1B+WfPGG2+gq9A0ra2thI1mOvroo7HTr0WLFhEhAuKBBx4w51Syxn63ZR8hUUlNTQ2jZA6dyJrbb7/d+ZtKc7LGVA5cfvnljDnhMVD2hRuQrJGsESKqSNaki2SNgQLYv38/G7T9y3X33XcfCZ50CxyQWdE0d9xxR3Z2NkkXoWAPJOyTKYzIGvb3cePGxdJuB8aPH2+fTFEPRewnyn5OPPFEaubAZA0Jm1f7P70Tee655/bu3UtUTijg3EVZg4xwsgbVZcURENSGpqH7XELuoDCsX44BAwY89NBDHCSVNVhMCyJr7AEPLTqHRFmDv323hl47fydr7FkUqihxoE466ST7yf0tt9xC/JI1kjVCRA/JmnSRrDFMAdQd+D8T3nvvPZI3GbexsZEsu3PnzqlTpy5btoxNHBFAQiUTk4DthznPPvss6RlBsGPHjscee8y+cwM//vGPZ86cuXnzZnQPaZ56SMb4TJ8+/fjjj8ehf//+N954Y1ZWln3X5Oabbyadu6pmzZp13nnnxWpq/2nSb3/72xUrVhAVQYL7Em7ivzLscJ2yf+7Pno6YrEE8WfG3336b4sgaMhOzb519//33kWjIlJEjR/7ud79bvny5+/YMI0Al9N2Kv/HGGwRgsubMM8/kFmLEaNE50BdkjYt2z549+BMJx3feeafzt4dPyBr8MRIeYTz11FM2mAwUYbiBkqyRrBEiqkjWpItkjYECILXbY4Zdu3aReu2jmZaWFpJuUVERRlIyGZ00jCevHOfk5NiXXu0LIogSFAw6ZsuWLVu3buXVvhnD1s/V+tg/90L9WDZu3Ghyh+SNguF4w4YNnJLUkU1MhKuKejgA3HJzc2mUekgkHOzevRsjAgJLJ7KGq1RL/BQnDNSAabW44sw7/TJFQnMW4fbt2ylIkBYhwRA/wosuUxw7A4U/44AbMdNB6qdCHGzEysrKMPLaRX/6RQzYuWrfKHJh8HrppZciayZPnixZI1kjRCSRrEkXyRrDKQDSKiKGrN8Q+w8T7B/qtcSMEQd7sIHcQQOVlpba0wh2dvwZQFNF9iVfXknkpGccuEoRqkIt4YOdq6gHnE0bkfXxpAY0h7+qmKRp/8ddcLAv2JLOLSROkQiIDEvwSWUNdq4SAM5ES8xoNUA8JRbnlUvUjJ0IXbtEyCvBkMMYBCohPDqOG8NiKofaTHLRTSrEAQs+9Bcjr130tw4yAjREuwwjsoaxQoGtXr3aPh2bP38+fcHt8LxXQbJGiKgiWZMukjWGS+okWtI2aRX5wlCAKRsTLuRgvwJwztjxx41T8rGlcPI0Wdw+tLLHIVYVPiaegPRMmkd2ABndHgXhTBEK4mZV8cqxq8ocqAeLC9XrSUewcxUfnOvq6mg9VXE65frlegEWoYE6YRyohDitOCNADdTMKZcYB/pIheaAnYao0DXXFX9O0T32jx0/8cQTCBpEVW5urv1A7Pjjj0dyER5hpOp15KHjdF+yRojoIVmTLr21P/ZBLKmTU4HEzzhYpucAu6kBMzpnLM7Z3DhlDNnoyd+8kqSdEjLMBztXgXSOP4kfLMf7qzK3uKrMwdQAFn9UiWDnqsUJBy3uIiQq9EdihPjjYMWBU3DHXLIWnQMHh+oPtLVixQr7gdiQIUPOPPNM+98VjjjiiHnz5iFxEFgEQ0EL+3CDjjMpkjVCRA/JmnTprf2xb0KK9eNZO9o9UzJnO2CLB0vPHDCedtVwPn4Hh/PkgFPnBnFVtdcS87EDMybFHBx+Y9LiZqE5a93addjVTnA1dJFEf1pB1uzduxdl8+tf/9p+cI6g4fjzzz/ftWtXSUkJqRRVhHMs5MMOhkiyRohIIlmTLr21P0YbS8/gnSfD8zhYYvaceiN/ew33eNO0iGQhWZaVleXk5Gzfvt2+h8TBnj17ioqKamL/D9fhfKNK1ggRVSRr0kWyRvQ1kDXchAiXurq6ioqKwsLCvBgclMf+x6jm5mbyaM/rrb6DZI0QUUWyJl0ka0QfxJRNa+zbRYibmhgcNMa+Un2YaxqQrBEiqkjWpItkjeibmLIhX8Z9xRjjYa5poLeWrWSNEGEjWZMuvbU/CtEVUDBxeBcObyRrhIgqkjXpIlkjRMYhWSNEVJGsSRfJGiEyDskaIaKKZE26SNYIkXFI1ggRVSRr0iVxf2SH0jcYhOizsDxZpJI1QkQSyZp08cuaqth/77w/9u/3x76dKTIYJtHwzkVUYE5ZpCxVFqxkjRARQ7ImXdgNPVlz/hC2SP1F4K909KCSUYNKRh9bctbg9r9R7ZY4H/1F5O/8IZI1QkQJyZp0cbKmuLi4pKSE93/2HynXi8yEuWMG23Xqb0dY4qnasdX+LTvPQ2Q4NsUsVRYsy1ayRogoIVmTLuyGLS0tpD22yKKiotLS0vLy8gqRsTB9pLqcnJzi8WdY4slZ8zkzW1ZW5nmIzIdZZqkyrSxbFm9zc7NkjRDRQLImXb6N/beCDQ0NvOdjrwRv4xSZCfKlsLAwOzu7aNwwSzzbli3Oy8uTYI0YtlpZtqiKHvvPzCVrhAgbyZp0sW8gtra2NjY2ukfc+sAic+G9O9lu586dTtZkL1/Ce/q9e/dqWqOBrVDgmGXb0tKCqpCsESIaSNYEgPtthfvPd0Tmsm/fPpRNTk5O4WVDLfHsWrW8srLS8p/nJCIBC9b9btFbzCEjWSNE2EjWBAY7o4gA5DnSTF5enntak7Pmc4QOWdDyn4ge3hoOH8kaIcJGskaIDpBgGhoa8vPznazJXbuqtra2x75+ISKMZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHJGtEeEjWCBE2kjVCdECyRoSHZI0QYSNZI0QHEmVNzprPJWtEIEjWCBE2kjVC/A+Ey/79+0kzeXl5Ttbs+WJlTU1Na2vrN9984/kJ0S0ka4QIG8kaITzQNGSXr+6d4FKO/6983NC2pkY9sBHpIFkjRNhI1gjRjmma5ubm6o1rXcrx/1W+9ixX8ZGyEd1GskaIsJGsEaId+/ipoaGhpKSk4NYrXNaxv+JLflr079z6+np9w0akg2SNEGEjWSNEOyZrEC4FBQWbFsxzWcf+tj/5IOlHXxwWaSJZI0TYSNYI0Q5ihdTS0NBQVFS0fv36nZP+n0s8hb86dd2qz5E7pB+kj2SN6DaSNUKEjWSNEB7ffPNNc3NzVVVVdnb2qrmzXeLJenTqli1bysvL9+3bpx9DiXSQrBEibCRrhPD49ttv29ra6uvr8/Pz16xZs2PiJWSdwl+dunr5stzcXH0CJdJHskaIsJGsCRJynshoyC779u2rqKjYtm3b6jdeKR37/U2P3bN58+aysrLGxkb7BEpED28Bh49kjRBhI1kTDOyM33zzTdV1Py8bPUh/+tNfBv2xbFm8PSNuJGuECBvJmgAwTdPa2soWyQ5VX19fF4MDkXHU1tZWVlbm5eVt3rx53bp1mzZtys3NraioqKmp8TxE5uNfoSxb+yeke0DZSNYIETaSNQHAbtjW1tbQ0MD+SP4rLy/nVWQoTF9JSQmyZseOHVu2bNm+fTuJp7i4WNMaPZhTYNmyeHvmi1OSNUKEjWRNAPA+r6WlhXfz7I9FRUWlpaVKgRlNWVkZOqagoABxk5+fzzEW75qICixSlioLlmXL4mUJs5C9JR0akjVChI1kTQCwG+7bt++rr75if+SNflVVVW1trT6EymiYQVLd3r17eeXYs4qowPJkWlmqLFiWLYu3Z369L1kjRNhI1gSAX9bwLpBNs7m5uVVkOLx9N7xzES1YpCxVFqxkjRBRQrImAPyyhvd/TU1N+/fvx/itEKJPwvJkkbJUWbCSNUJECcmaAPDLGtsf2aTYOr3LQog+BsuTRRq3bCVrhIgAkjUBkChremB/FEKkQ68sW8kaIcJGsiYAJGuEyDgka4SIJJI1ASBZI0TGIVkjRCSRrAkAyRohMg7JGiEiiWRNAEjWCJFxSNYIEUkkawJAskaIjEOyRohIIlkTAJI1QmQckjVCRBLJmgCQrBEi45CsESKSSNYEgGRNJIn9a7TxeNdYOTFWrlzpnR8GWJf9g5DRSNYIEUkkawIgkP2RbHHuuedapkzMHBi5hEOGJhXCnjFjRiwt/o+xY8fefffdmzZt8pz6BoQKK1asmDBhwuDBgy3UI4888rLLLnvuueeqqqq4yvyaHTeOsXiFe4S4kWQYZ86cyY3nXQ4Bf5dJwO0D5MNzOkS8wr13PweybA8VyRohwkayJgDS3x/Z3NnXzjnnHNLGsmXLKO62ew44Xb58OZdwsKRilzIF69306dNjaTEJ48aNCzUrdx1CraystIlICpfa2tpaW1vtdMmSJZz2QDo07GawkRwwYMDZMSySU0891SSX5xoc1qjrcnNzM11mlJBTnE6cOBGHQ2oXZ0LtdvGgoFOSNUJED8maAEhzf2RPZ1Oj1JgxY9joP/nkk5aWFlcDV8kiixcv5hIOuO3fv79X0kD3cL174IEH6MLIkSOLioqKi4u3b9++YMGCW265BSOQlbuRVOy5RVAPsQhgw4YNyAXq7N+//9SpUz/44IOcGNhnz5591VVXXXLJJU1NTQ0NDbGo++Fg/7Npz8yI3QwPPfQQTY8aNaqkpKS0tHTnzp1XX301FiRXGJFYo67LNTU1dJl2TVFdf/31XRF29iSS+aI2nMvKyg6peBjQaDrLtntI1ggRNpI1AZDm/shGz1vh2tpaUj4bPcmeFGL5CUwTLFq0iEs4kFT8oqfvY72rq6tDJdCFYcOGbdmyZdu2bciaXbt25ebmIuNMSUyfPv1Qx+3RRx+lIO/7GSVOvQvdgqbLy8tPOeUUKjzhhBPWrFmzdevWzTE2bdrEa3Z2NtHiw0xVV1fjBu+88w5do4Nptt5F7Ga49957afqMM85ww4jqsngQOoErG0aGW27v3r3WBIKGm5BBqKqqYjQYioMKOy7ZsxmmGE+Gizu868VDgn6ls2y7h2SNEGEjWRMAae6PODc3N7PLn3nmmWz98+bNI2dYpgT2O3IA6ZNLOFRUVPTM/hsUhEpSZFjuuOMOunD66aeTg1E2iAawxPzWW29xCchzXUxvuPEu/+GHH6bUmDFjGMB0lA0Fyaz2FGTgwIErV640KUOEqBmDOHNyckjqZGJmIRZvPyIn3/eM0HQ3w913303TQ4cOzcrKsjHk1eL54IMPGIoAg6FRauOWq6ystCby8/M5hoKCAnRecXFx58IOOzNlz2b+/Oc/I2KAQSssLOxK8fCwfknWCBExJGsCIM390YqT0YcPH87W/+abb7LpO1lDuq2vr58/fz6XeINOWiUr2Htfsu+MGTOOO+44jn/yk5+sWLHCigBZedy4cdi5OnPmTCxeY//9LxG6UkceeeTEiRPJLn4H6NzHfX9548aN1go+U6dOpZQ5+KF3JFqy4O23344nyXjnzp1s5WRH6uTYErNJuqeeesqpE16JgZ4CrVjv8AHqxM2OHeeccw52fMC6b2WpxLqDD68WlR/8qY0pGDRoED6PPfYYxdExiJiioiIGvKysrKSkhATMAVXV1NQ4WfPGG29gMVlDPYsWLbJGYe7cuVj8XwMHnBlMhsuNWCq7F5wP3OxmmDJlCnUykgTJGBLnqlWrYuH0YyTJkcgIO6VdwqBOptIaoh6KTJo0CSMO2G187JIDi/M59dRTFy9eTN9jVfZj7ug+jBgxglOiNRXOCFCK2rgVzRMYBybOOznAgw8+iGdicWIgWlecsoynC4wDs/MGwMaKY2aZ7pjDoUKLTDq1lY4exP3Z2NjIuHEnhAo9pb9MmWSNiBjsGCxJ7mfvvCNslVzl1TsPE8maALD9kZ06HVlD2kC12MQnypp58+ZxCQfyK+/X7Tut5Bte/ezevdt2T/cTHmPChAmWNtjEE0uRIRAoXCWYzn2oBOz7yzRhqcXhWrF+GVj8sgbpRhfoIJt7dXU1x3v27NmyZYt9yeaaa64xiZA0BvelYwaHOu3YMWbMGOz0fc6cOZ7pAIRKbBw8+uijiRFySj6rq6szZzTNjh07EDSER76xpwvkPIYd7IDumLPJGntWZE34YZpsphix2LTE+9BHeprKnhinX9YMGzaMHQQ1vGbNGhOF1113XV5enn1MGaum3zPPPGMHf/jDHyhL35nEuFkDmqNH1hyvCIVEn2effdYOaBRnem2fmd57770mpDAmThndX7p0qXdygPvvv98VnzZtmhVPHAQDBUPYQPxmiWuFyeVq3Fh1BUoxs+izkjHHOZER9l/xOd/fc9Z31555rLMwZX1c1jC23Rjevg83z4svvuidiLTpGVlD/dTDm0/vPBmSNQHA/thjssYe2tuXi+GJJ56gRTJH//79OeV9MFUtWbKE44svvphM89lnn5100klkPiqkKsscJ5988oIFC2iR/H3fffdhITdY2oNOfKgE7PvLQM0bNmwgMVx11VVm4So1+DdBTv2yhgRMwiZIPLHTtdLYl14nT57M1VGjRtmbZothwIABzz//PJmb4mxA9hUcoBSqiNjcEyD0HD4IDqI1n4suumjt2rUYs7KyGAozPvTQQxahF1wMi3DXrl3ms23btsLCQsQBRsaEfIMDr8ApsRE8XTBnJsu68+qrr5qFNE8MtLto0SLG0IyffvoptQHHgwYNoi0466yz/vnPfyJBEu2zZ8+mobg43c1gsuaII44YMWIEU9DeQEzlrFixgjXPzcM4mPHYY48lDPQZw8XA0haTiJ0BWbduHQPIEP3qV7/Ccv3117sbYGzsWSCj/frrrzP4MHPmzIEDB7bX2K8foocu08ro0aM55d6gOWpGOXFKKWQlc0op2uVeBRqyJ5E33ngjHUTLMj7MtRUnMCblrrvusuL/+Mc/8Aem3mYcOYsDbhybD53irsPTLNzkhO2/67oCE0qdhJp/1mAnMsL+Q9bkjPnul2cemzVicNYd121/5A5kDQNIMF5YfQmGlIHdt2751/Nn7d/X1I1B9sPU23zBFVdc4Vl7D8LoXNZw1aI12H69CxnCJ5984oXer9/dd9/tWTuCT1Bz0bmsCYrPP/+cViRrQqd95feUrCHj0oRllMsvv5x7iDRDUrcEz71LLrHH/uQh0tKePXusCDv49u3bsSOAli9fjpLgFCh75ZVXYieV0ij2TnxmzZrFO8uFCxdyDPhY61YKCJV87N/76B1pmzTmZA29s0cywAECIj8/32QNC4wsSIuxytq/KcJWSOX0gp5++OGHZresieXmm2/m9Gc/+9nmzZsLCgro+zXXXIPl/PPP9xek/gsuuAA7SZQ36KQQf4SW3si17VXHnniZUolza5+M2C5PdxhPc2ayTE4hIDh9+OGH/e1u3LjR1AACEWFBT019IkZtXmiIEUu005GkAfhljXHMMcdceOGFFGQETJAxvLRlV//2t78xQSROe0LG9GE88cQTic0/uVSCPTs7m7lbv359rOj/Bh8HonK6jQrpMiPgHrcwZbRrV7k3rGbKMuxME/qDAJCeXGXjY0yooaioyD3soUeI9VjpfugVf2DckxjRMfg4Kfn3v//d3XX2mOrjjz+Ou+u6AsNLtQS5Z9T/np2E/Yes+feY720aOXjpsGM+eWsuo81Q2D3phdVnsLudga1d+AaRl48bWvf2S21NjRg7GWqbI4OUyY6UmFCRC31B1nQOwR999NHeyaFA17z+++jd/iadBSPAuZCsiRSs8/RlDQnAZA33WSeyhuSNp30v4emnn966dSsJgHxmCR7pQIKB3/zmN5wC7+nvvPNOUgu5h93fjEm5+uqrSc+d+1x11VVkFwvm+OOPd62Tq8yBq2R9f/c5TpQ1rnfs5jRKVrO+P/TQQwygxYDnli1bqNlaIYeRd3/0ox9xyfJiTk7OTTfdxCmyBjdyJ9LQkiXpMK6gfRwzdepUxsGfAjlghNFqDJpVTtZEFqRKk3QHKeZkzdy5cxlbdImdJgZ83XXXYWfEmDXU2F//+lfzBFQpbigYwn7yySc9a8yOJkBpdS5rTj/99E2bNtEKMAXAgBAMwbvw0DHYUQAMF8b777/f7El54YUXmAv7FpcNvn1peseOHcRJW0cddRSXrBV3ExIMk25Kd/jw4ZQCGwGmCTlF79jpTNZMnDgxKysLkcRkmSJhRhgBE6xDhgyhII0a1MMAYofPPvuMO9+OV65cSeX0i6EeNmwYFlqPu+u6AoPJzUAwX5x+1OLFi0ljn8bgIFS4vZGMH330ERs0HWQwkTXcb0x3X4N1yi1RMX+2k2UmblobGxjtuAVC15gLbl3v/L//RRZAhsqaNINkKLyjGAxCL3Y5pmqCkTWs5fZFeADuYe+CT9bYFwqBA+9asg+hzGIcd9xxnvUA/qtOxHDgmWKkklCSNQHACg9E1thjfG4ONvo4WYOC4dJ5553Hm0uyo6UE8hBpgDQD9t2UyZMns01ziv2ll1667LLLkDXY77vvPrLd+++/z3EqEATkpwULFnjnySCTkSDJ5RyTVl3rHJgDaYx9kA3RbXn0rnNZQ/6233kRKm9e2eUtTkQGKdCyNSmNO5hkhpbiElsnSgij9ZpIuERgtG4j89RTT8UVRKxgR+EhQax1C48DG2E8p02bhs8xxxzDIHdd1jAdxGynZN+4du3pEW4WMGmMEb722mvtKc7o0aOpikt4xtm5JfzDCC5U+7zmtNNOW7duHemfFlEeaAhuIfI04+8ebLBn0SLtMv4Mu/0yPBWmMEygIBcQDURFtdYXWnGyxiSafa7EkBK/iSGmDDfuQAaQ6UDQ0DvqdLL1j3/8I/FwFVxxanvvvfesOFIGlUYTQKc4xQ6IACbdjv13nakl7p+4u64r4G+yZsfInntaE42/pE9ukj7bYN4TE2onqdTuE4dn7Yjfh6po16+lgMq9y7EH2Pgjubxrsda9ayk+hPIXd1BJ3FV/F4jBjK5Cju3AERenqUDHQSOJ64gZXSmumiWpfMGYaPePg5+kkRh+qeFwmsNkjdM0htMrcbLGnOOwSxAnX8BakazpOUyXpClrSI3PP/88U9W/f3+2WvuYhi2D/Mrme8IJJ3AJdUKOcckb4cIbcZIHkB6w3HzzzawcdnySB2mJnLd27Vry9IABA8gu9nsZ0uHq1asRELz/xofU+MUXX2zYsIFGST/2AVZSH05JOdxJ9p1c+0aLtU6LWKCwsPDrr78m+7rNjl4klTWudzRkH8EQvOVm2opV1u/tt9/mbT1JGi1CDYti/3gPuHbtR+PkYCvIyNhjKpaWvyAx22JjlJyosvA4IL3Z0xqSqMmmk08+mcqdjx/C9sua119/nUEDkyPoBn+7yDLb78yNTM8A2rwwTRdeeCGXGHAkQqKdMU8la+xpDXPErJH76R2V0yi9YKi5l5ysmTVrFleJlkvcRfa13xtuuCHVDUAkbrdFqXCzURZBTF/efPNNszPXNtSmS5hWNAp6zq6iiriTKUKcNNoU+6cLqcTuWDfFBGzFmUGGhTBipfuhb6gZf7qA3b49g94lNjSW+fjvOntaQ6hxd11XsMEknp2SNd36Q9w0bV3PimAwyYVJE2pScPZrAj9+/cFaYHL9FiC1+/OuSRy/XKBmfySmDOIqMbjUSQrvJMi4Joy4wGjUOzqAX/bZKvOHTfG45uJa4TSuI1ji4sc/MTBIZYdOupkIAYB7QmOfB4HJC6dU7CrYrmuKxC9rXMGYVzt2lRrs1K5atXbq5JGVdVoqKZI1AcDCTlPWkI3Yx9mv7XOQk0466d1338UOpB/7gjCbO7qETOOe1syePZtTMijwVhuLPa3hrS0O48ePJ0+z4yMaUDbkEu6SSy+9FDcqnDt3LkmUxPb3v//99NNPf/TRRylIaidVdOLDOqQSkzVkFNc6BbEAFlJF57IGgYUywPL+++9ff/31VvCyyy4jWhIkdnIewWOkyzNnziQvUgMZ2tQPECq5kxGz5xZEQgCckhHdh2i//OUvt23bRkGq5diMibIGUA9kX0YVLbhs2bIhQ4bgiRBEPfh/mM0xW8O4ceNssqxC9ArZl14//vjjZpk2bRrB4EMkJkaBRUsaRruwgzDFSBkW549//GMuIX2Y90Q7wcflaY79sgZZSYWMFeOJkbvOPsjwh/fqq6/SLxtAfAjMnriwfSAgmNzly5dPnz79hz/8IaqFqac2psC0AoOPqmhsbKQsg2/fv4E4XcK0tiuDnTvtp/7MER1nfhlk4gGOUTn2idUtt9xiM0VUrjijRw32aR3FEetMB6P6l7/8xWYcsUj9zA7H4L/rLNR58+bF3XVdgbEiMO4lPa3pxl/R7b8rX/c5t4cNO0ujE4kQR9dTaSwdd8jH/icWSUlM9qno3LOTIE1veScxCMmvWiDOgar8HYl7cmPg07lqiSPRIXG4jFR26KSbcZjyiNMTfrFisoYdzC4ZWOyjKL+nPXRxqsWw4hx0Llwka3oIMnc6soZNga2fdMvm/vbbb9v7+zjY37nEW3O2exKkyRqSFomcDR24a7Ega6hkzZo1TgEYvCcmGbCDL1269Ac/+IFn9UEuJ8mRbCjOO+9UPryrJpfYh1DsL651kpz5HFTWJAVxQ82kLrrGOFDt6tWr7cGJH/cbYxIwFTLUluCJhGHhlIK8p3e/A3cgL+wDvqSyhgjtAQzjg4ZDW5x//vlWMJFzzjmHpJsoa1AhJgf9/O53vyM2DljMBEwmPvHEE+2SQbLHTkiJdoxE5b+RiNkva8joSBDqZHhRD3ji4Ppi9RAe9XND0mV7IoV8+c53vmNX/fzmN7/hKnqFvnz44YdoGu/CAV555RU74A7xy5rbbrsNC3PHuJko9zN27FjiqaurM1lz66232kwx0XGyZuvWrSaM4sCIMPXLGv9dZ8PbPVmDP4GhKVefdhTphF7zfgBNxmsPQEP2VZ7FfRiG5YMPPlg5/W6/oNkx8ZLVb7zCgmXi0IXcWtx1B83BfrqeSpPmY/8OyQ3gWX3YXWHEqQ0/ncfceZBc8gcWdwpe8wfwX0XQJA2b0Y5rsfOOJMbfPljJ5EsqO3R9LkyLxKkWpAlGExlOl/hB09iDlkQBlBQq9HsmIlnTQ7Cq05Q1pCV2WPIK2XHlypV33XWX++Huscce+6c//WnVqlVs7uRd8kHFgX8Jbc6cOZySroC7FguyhtsCB6SJfXmWxXDPPfeQErCTPKiE3eqmm26yD00AhfTkk0+yQ1GK4Mk6JJikPiRRmgO77UhLrnUyjXnShbgEw1CkkjWjRo264YYbWMyICTpODORyihMJK3/ZsmW8uaeno0ePvuSSSxYtWpSXl2cFGYc4WUOGoyzploJsuOTgCy64gIJjxoy57777srKy7JdWSWWNjb+lSfvwjmDeeuut3//+9+4X2kBVNlD0l9bNaF+aAfIurTz++OMWMM4vvPACAdvjBEaMIuQAkqj9pgzdcO211yJAGbrq6upEO4MQdyMRp8kaezJnvW448P9smAP+fllDuxyb7uGNNbKVrr377rsoMBMuyF80HG70iwgZHHxQz++///748eNNHDMICxcutP0LTNYQttMldJMZ4bYBRtj+VUPj7LPPphfuaQ2yhpjpL3eOFbcPobi10KPcnH/961+dwkOM/uMf/0Btc4n6GWGz++862/fnz5/fDVnzn9gvoejXuqFHcbOxZJb3LAjBPg7DwvJcNWOaCZqt1/3yo+efRv+RWphrppIBNEnN/tOJRIijk1Rq+5ifVPnYoKqkEsFB/KmUTaIs8NN5vvc/sPEfO/wWIvQ3REjtHUtGJ31J7IhkTSokawKAVZ2mrLHPQcgWpDc2d5IryYPtHsgZpBk2d4yIBlICTZAGOMWZ5EdiA/Z63Eha3BZkJnZ8CpIJKEieJjGQt6iNIliQEZYqAAuvXKJmpBW5LScnJ6kPrXMVqJnWWcz+1mkLf+oh3ySVNThsiEFZqxYLm6N1jTrpF4OAc03sJ9+MA0KBS/SIprHQQSx0hwirqqoIGDsO1EOXCZ6sTD2dFGRUqTxO1oAFSXGcGUM8qYRq8WfcqIRXLtE7OoIbHScGmuCVvnNKW520S2fxIWCmGDs1u3mxOqnEb+cA/6SyhuGliPWaUowYN4/nEesIsoaGiBwfXjnGYh9OMcKpJpc66ReV40P8LjzrC6PKMX2hIHUSGwNOQRsB0hs+FjwTip1SjJgpNupE1uCAMwOCkYYYBxq1COkFUE+7LNq6lfsKI6c4M6RMB6cUx8JxF++6rsCYUEn7YA4/mn7RNFVRoTAYfwZ8/fr1WX9/ZMfESz596TnU7ZIlS9DcNub2pJBhZNi7nh0hlTPGuMzdST52JGb3OFJV0nnBg/aIq1Ytr4mepF7vKAZ6hVvXjjnoXIqlItaP/3UkMf44B0cqOxy0mw5TG3F6IlGssPbtkoGlix9COToXLpI1PQS5JB1ZA/iTa0nMbBZsKOzmbLLsKez7HJAtOOYmIJdbkiA3sMs7KQCkRnZ/MoqlE15ZPJS1/dpSEc5cwgcLUKc1gQU7rVMPb7/IFkl97H02OYkDclJi64RE2caOv0x2iZarZMr2XTIrix7RR+sm/SVaq4p3fiQnDoiWJnCw+PGhO2ymWAgGf8JgHEiuuNE0AViq67wg2ZFu0kpc/uOUmGmXMJAXeFLcFIk/VBsommYiiIGGLHIisTyNW9J2qZMIKcX82tjixiUbMWImMP98MdcmR/w3kgXJ8JoIoNdUSF6J8+FGMhmBD68c28cE9JrxSTW5dgNw63KDmbym4xYM6Y3u4EbA9Ig66TLdoVN0mVPCAIYCTyvFK/5213FHUbObKSyc2jhjYUDoKQmSwDilOco6rBIu0RwN4dDFu64rMCY2mHkjj6FaDqiZySUeYTBZTBPz/sUXX5BO1q1bxw1jNzP3icllN+YoEu5hO3aQNRNTeKpUSrryjg4Ql48/SfZvx/mzOwEkNhdXiSNRFvg5aL6nLQuY18SOx/UFB79i86ucpHSlI4nxxzk4UtnhoN30Q6fAPbAxhQEmUNwzGLsKx8X++x2TIH5ZYwXdt4CN9pIHytqx0z321WM75ZVjyZrQYX9MU9ZYxrIHFSYayB8u65BU2ErIHKQlthI2dHYc8oTlDCxAqmBfJjlhxJMDdnzWBmW5D0hUeFLQ3LDYJaAtS1RcJf+5qpL6sJcBB4mt0wRGewMXl2hJqHhaYqNfQJ1UTkIiF9IWqZdBQ9BQEPCnFSokBhcAxwwC6YdIbFdlNEhCVIsnAVgknRdkZOhm0vyHhQAIw+k2S+QkaZenqYqmLU9zQNNUSIuUwthJu4wVfbTkjd3NC6cYaTHOjj+9S3z8wODYTeJ6bZLFu3ygF5QlMAvP1YMbzsSZanIZGUbe+XD/mAMSh2OKgI0hDnSHIN0I2Fy4W9duWtN8SAewmaJaCnJKDVYcB8rGDSDFgQipwQYcDumu6wr40y71F48aRD1USyV0Xzi4JRhbm1Zw08FA2WqNuz/JN8yadxLLu1i8Ex+pUimJ35+nyesU9+djrmLxl437aMY+3/ELAuJJGgMkygI/Xcn31sGkbomNxmkLHIjWO4nBVRd5VzoSF6EF42/CEWs5uayxat2sWbuphsWkSRxOYZisifuAyWkXv6yBpJ9DuavU6ZkOgL9dYovwTKmf90jWBIDtj2y73ZY1wAZhyoZdg1SBLiGFAG9n2XPZXCzr4MMrx7iRD9jc2V+AgmQXMwI7PpVQkDRARsHCJoUb/hxjwU6OAfYpMo1VTtjUj2cqn9he177ZpWqdvnPq3+ysX3hylcEhJIM6EWfESZKjTic1eCUMgrFcSAw480pZBgGsCBW6SGiaAKgBMHZeEIdUs+NCpcL22SwrY/xNmvDK5m4pnLaA7tO0RUKXO2mXocOZIgyOf17ovtUWZ6cS/JPGSYS0hb+/1zZuho2eBWPhWT3Y3aVUk2s3QJyP6wunbgzpsn8E8KcsFk7pBXUiNShIEbsKbqYIntO44jaArlGKQzp3XVegpxSka6WjBxE29dtgCoPxYUBsWm327Zaz+wQHbxw74iWcGHH5Hv3hXeiIP7t7phgkV5IxB0644EmdZjT8mgZIz0gBCnqXY3jXYlgWT4rn0fGLuo6k2sVqi1MnJi8cnjUGp/6A4wbE38RBO2L424pJlw7DZfZEnIgxTMoYceOZiHtCY7gnN2BKhQN7uAJOi0CcrAGzOOI0iv+qvx5wkshfm58kIyUOFbc/piNrwLYSyxC26QNbie3sbs+17QY3cEZnAbZ4/KmESMDSgO1EvHKMBTvVAm7+yqETHy7ZVWsosXWzeP05gHNwIQHHNGFFqDOulLVCo553rBecGtYdiGsXulIwri0/VgMVUoqOM/gkcoMZIWZrC2wQXIUHbddKcewGAYdO7KnixIh/e59Tj7YF4w8v7hJN0FDc5HLViPPh1SarvSe+vlgM1oTBKZ5Wrb8j4Pw58J+ag2uU+hMDay9/oEeAxeL018mp9bHrUCdtpb9sI4wb5NjMH3z5HG4gO/wPVEQnJMqa8JCsCYAA98f23Tq2xbObsIkYbh/3nA64GYkWg0ocnJobxF2CmHuHrapzHzs2UlnisEteXT7M7jl1xC55fjHMkgqv2MEKek6dghulGHawxAkcx1XlcKUg1qCHWeLwrsXwTDE8UwxOrc6kmL/hmTriXTuAZz0AFq+ZAyS6pfJJilem05rtwEg8NezUKxnDLP6rRirLIUH9kjUHxYY3bi6EcfTRR8c9qhGpsM+V/E93wkOyJgDC2B9tEzE8k+hxvAk4gGcVkUCyRqQDguagH9kI8H9ulerbMMEiWRMA2h+FyDi0bEU38H+zx0j1BVthuO//dv7zpQCRrAkA7Y9CZBxatkJEEsmaAND+KETGoWUrRCSRrAkA7Y9CZBxatkJEEsmaAND+KETGoWUrRCSRrAkA7Y9CZBxatkJEEsmaAND+KETGoWUrRCSRrAmAxP3xP936Z0+FED0Dy5NFKlkjRPSQrAkAv6yx/1xm/4F/ZVwI0QdhebJIWart/4mVZI0QEUKyJgD+J2vOH8IWqT/96S+T/s4fIlkjRGSQrAkAJ2uKi4tLSkp4/1dbW1tXV2f/V6UQoq/B8mSRslRZsCxbyRohIoNkTQCwG7a0tNTU1LBFFhUVlZaWlpeXVwgh+jAsUpYqC5Zly+Jtbm6WrBEiAkjWBMC3337b1tbW0NDAez72SmkaITICW60s26+//polzEL2lrQQImORrAkA+wZia2trY2Oje8StD6GE6LPYCgWOWbYtLS369aIQ0UCyJhjcbyt4z4e+EUJkBCxY97tFbzELITIZyZogYWcUQmQi3hoWQmQ4kjVCCCGEiAiSNUIIIYSICJI1QgghhIgIkjVCCCGEiAiSNUIIIYSICJI1QgghhIgIkjVCCCGEiAiSNUIIIYSICJI1QgghhIgIkjVCCCGEiAiSNUIIIYSIBP/97/8HMEHnJlJrnjsAAAAASUVORK5CYII=" style="margin-left: auto;margin-right: auto"/>
|
|
|
95
|
+<p>Importantly, ClassifyR implements a number of methods for
|
|
|
96
|
+classification using different kinds of changes in measurements between
|
|
|
97
|
+classes. Most classifiers work with features where the means are
|
|
|
98
|
+different. In addition to changes in means (DM),
|
|
|
99
|
+<strong>ClassifyR</strong> also allows for classification using
|
|
|
100
|
+differential variability (DV; changes in scale) and differential
|
|
|
101
|
+distribution (DD; changes in location and/or scale).</p>
|
|
|
102
|
+<div id="case-study-diagnosing-asthma" class="section level3">
|
|
|
103
|
+<h3>Case Study: Diagnosing Asthma</h3>
|
|
|
104
|
+<p>To demonstrate some key features of ClassifyR, a data set consisting
|
|
|
105
|
+of the 2000 most variably expressed genes and 190 people will be used to
|
|
|
106
|
+quickly obtain results. The journal article corresponding to the data
|
|
|
107
|
+set was published in <em>Scientific Reports</em> in 2018 and is titled
|
|
|
108
|
+<a href="http://www.nature.com/articles/s41598-018-27189-4">A Nasal
|
|
|
109
|
+Brush-based Classifier of Asthma Identified by Machine Learning Analysis
|
|
|
110
|
+of Nasal RNA Sequence Data</a>.</p>
|
|
|
111
|
+<p>Load the package.</p>
|
|
|
112
|
+<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ClassifyR)</span></code></pre></div>
|
|
|
113
|
+<pre><code>## Warning: multiple methods tables found for 'aperm'</code></pre>
|
|
|
114
|
+<pre><code>## Warning: replacing previous import 'BiocGenerics::aperm' by 'DelayedArray::aperm' when loading 'SummarizedExperiment'</code></pre>
|
|
|
115
|
+<p>A glimpse at the RNA measurements and sample classes.</p>
|
|
|
116
|
+<div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="fu">data</span>(asthma) <span class="co"># Contains measurements and classes variables.</span></span>
|
|
|
117
|
+<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a>measurements[<span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>, <span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>]</span></code></pre></div>
|
|
|
118
|
+<pre><code>## HBB BPIFA1 XIST FCGR3B HBA2
|
|
|
119
|
+## Sample 1 9.72 14.06 12.28 11.42 7.83
|
|
|
120
|
+## Sample 2 11.98 13.89 6.35 13.25 9.42
|
|
|
121
|
+## Sample 3 12.15 17.44 10.21 7.87 9.68
|
|
|
122
|
+## Sample 4 10.60 11.87 6.27 14.75 8.96
|
|
|
123
|
+## Sample 5 8.18 15.01 11.21 6.77 6.43</code></pre>
|
|
|
124
|
+<div class="sourceCode" id="cb6"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(classes)</span></code></pre></div>
|
|
|
125
|
+<pre><code>## [1] No No No No Yes No
|
|
|
126
|
+## Levels: No Yes</code></pre>
|
|
|
127
|
+<p>The numeric matrix variable <em>measurements</em> stores the
|
|
|
128
|
+normalised values of the RNA gene abundances for each sample and the
|
|
|
129
|
+factor vector <em>classes</em> identifies which class the samples belong
|
|
|
130
|
+to. The measurements were normalised using <strong>DESeq2</strong>’s
|
|
|
131
|
+<em>varianceStabilizingTransformation</em> function, which produces
|
|
|
132
|
+<span class="math inline">\(log_2\)</span>-like data.</p>
|
|
|
133
|
+<p>For more complex data sets with multiple kinds of experiments
|
|
|
134
|
+(e.g. DNA methylation, copy number, gene expression on the same set of
|
|
|
135
|
+samples) a <a
|
|
|
136
|
+href="https://bioconductor.org/packages/release/bioc/html/MultiAssayExperiment.html"><em>MultiAssayExperiment</em></a>
|
|
|
137
|
+is recommended for data storage and supported by
|
|
|
138
|
+<strong>ClassifyR</strong>’s methods.</p>
|
|
|
139
|
+</div>
|
|
|
140
|
+</div>
|
|
|
141
|
+<div id="quick-start-crossvalidate-function" class="section level2">
|
|
|
142
|
+<h2>Quick Start: <em>crossValidate</em> Function</h2>
|
|
|
143
|
+<p>The <em>crossValidate</em> function offers a quick and simple way to
|
|
|
144
|
+start analysing a dataset in ClassifyR. It is a wrapper for
|
|
|
145
|
+<em>runTests</em>, the core model building and testing function of
|
|
|
146
|
+ClassifyR. <em>crossValidate</em> must be supplied with
|
|
|
147
|
+<em>measurements</em>, a simple tabular data container or a list-like
|
|
|
148
|
+structure of such related tabular data on common samples. The classes of
|
|
|
149
|
+it may be <em>matrix</em>, <em>data.frame</em>, <em>DataFrame</em>,
|
|
|
150
|
+<em>MultiAssayExperiment</em> or <em>list</em> of <em>data.frames</em>.
|
|
|
151
|
+For a dataset with <span class="math inline">\(n\)</span> observations
|
|
|
152
|
+and <span class="math inline">\(p\)</span> variables, the
|
|
|
153
|
+<em>crossValidate</em> function will accept inputs of the following
|
|
|
154
|
+shapes:</p>
|
|
|
155
|
+<table>
|
|
|
156
|
+<colgroup>
|
|
|
157
|
+<col width="25%" />
|
|
|
158
|
+<col width="37%" />
|
|
|
159
|
+<col width="37%" />
|
|
|
160
|
+</colgroup>
|
|
|
161
|
+<thead>
|
|
|
162
|
+<tr class="header">
|
|
|
163
|
+<th>Data Type</th>
|
|
|
164
|
+<th align="center"><span class="math inline">\(n \times p\)</span></th>
|
|
|
165
|
+<th align="center"><span class="math inline">\(p \times n\)</span></th>
|
|
|
166
|
+</tr>
|
|
|
167
|
+</thead>
|
|
|
168
|
+<tbody>
|
|
|
169
|
+<tr class="odd">
|
|
|
170
|
+<td><span
|
|
|
171
|
+style="font-family: 'Courier New', monospace;">matrix</span></td>
|
|
|
172
|
+<td align="center">✔</td>
|
|
|
173
|
+<td align="center"></td>
|
|
|
174
|
+</tr>
|
|
|
175
|
+<tr class="even">
|
|
|
176
|
+<td><span
|
|
|
177
|
+style="font-family: 'Courier New', monospace;">data.frame</span></td>
|
|
|
178
|
+<td align="center">✔</td>
|
|
|
179
|
+<td align="center"></td>
|
|
|
180
|
+</tr>
|
|
|
181
|
+<tr class="odd">
|
|
|
182
|
+<td><span
|
|
|
183
|
+style="font-family: 'Courier New', monospace;">DataFrame</span></td>
|
|
|
184
|
+<td align="center">✔</td>
|
|
|
185
|
+<td align="center"></td>
|
|
|
186
|
+</tr>
|
|
|
187
|
+<tr class="even">
|
|
|
188
|
+<td><span
|
|
|
189
|
+style="font-family: 'Courier New', monospace;">MultiAssayExperiment</span></td>
|
|
|
190
|
+<td align="center"></td>
|
|
|
191
|
+<td align="center">✔</td>
|
|
|
192
|
+</tr>
|
|
|
193
|
+<tr class="odd">
|
|
|
194
|
+<td><span
|
|
|
195
|
+style="font-family: 'Courier New', monospace;">list</span> of
|
|
|
196
|
+<span
|
|
|
197
|
+style="font-family: 'Courier New', monospace;">data.frame</span>s</td>
|
|
|
198
|
+<td align="center">✔</td>
|
|
|
199
|
+<td align="center"></td>
|
|
|
200
|
+</tr>
|
|
|
201
|
+</tbody>
|
|
|
202
|
+</table>
|
|
|
203
|
+<p><em>crossValidate</em> must also be supplied with <em>outcome</em>,
|
|
|
204
|
+which represents the prediction to be made in a variety of possible
|
|
|
205
|
+ways.</p>
|
|
|
206
|
+<ul>
|
|
|
207
|
+<li>A <em>factor</em> that contains the class label for each
|
|
|
208
|
+observation. <em>classes</em> must be of length <span
|
|
|
209
|
+class="math inline">\(n\)</span>.</li>
|
|
|
210
|
+<li>A <em>character</em> of length 1 that matches a column name in a
|
|
|
211
|
+data frame which holds the classes. The classes will automatically be
|
|
|
212
|
+removed before training is done.</li>
|
|
|
213
|
+<li>A <em>Surv</em> object of the same length as the number of samples
|
|
|
214
|
+in the data which contains information about the time and censoring of
|
|
|
215
|
+the samples.</li>
|
|
|
216
|
+<li>A <em>character</em> vector of length 2 or 3 that each match a
|
|
|
217
|
+column name in a data frame which holds information about the time and
|
|
|
218
|
+censoring of the samples. The time-to-event columns will automatically
|
|
|
219
|
+be removed before training is done.</li>
|
|
|
220
|
+</ul>
|
|
|
221
|
+<p>The type of classifier used can be changed with the
|
|
|
222
|
+<em>classifier</em> argument. The default is a random forest, which
|
|
|
223
|
+seamlessly handles categorical and numerical data. A full list of
|
|
|
224
|
+classifiers can be seen by running <em>?crossValidate</em>. A feature
|
|
|
225
|
+selection step can be performed before classification using
|
|
|
226
|
+<em>nFeatures</em> and <em>selectionMethod</em>, which is a t-test by
|
|
|
227
|
+default. Similarly, the number of folds and number of repeats for cross
|
|
|
228
|
+validation can be changed with the <em>nFolds</em> and <em>nRepeats</em>
|
|
|
229
|
+arguments. If wanted, <em>nCores</em> can be specified to run the cross
|
|
|
230
|
+validation in parallel. To perform 5-fold cross-validation of a Support
|
|
|
231
|
+Vector Machine with 2 repeats:</p>
|
|
|
232
|
+<div class="sourceCode" id="cb8"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a>result <span class="ot"><-</span> <span class="fu">crossValidate</span>(measurements, classes, <span class="at">classifier =</span> <span class="st">"SVM"</span>,</span>
|
|
|
233
|
+<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a> <span class="at">nFeatures =</span> <span class="dv">20</span>, <span class="at">nFolds =</span> <span class="dv">5</span>, <span class="at">nRepeats =</span> <span class="dv">2</span>, <span class="at">nCores =</span> <span class="dv">1</span>)</span></code></pre></div>
|
|
|
234
|
+<pre><code>## Processing sample set 10.</code></pre>
|
|
|
235
|
+<div class="sourceCode" id="cb10"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="fu">performancePlot</span>(result)</span></code></pre></div>
|
|
|
236
|
+<pre><code>## Warning in .local(results, ...): Balanced Accuracy not found in all elements of results. Calculating it now.</code></pre>
|
|
|
237
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-5-1.png" width="700" /></p>
|
|
|
238
|
+<div id="data-integration-with-crossvalidate" class="section level3">
|
|
|
239
|
+<h3>Data Integration with crossValidate</h3>
|
|
|
240
|
+<p><em>crossValidate</em> also allows data from multiple sources to be
|
|
|
241
|
+integrated into a single model. The integration method can be specified
|
|
|
242
|
+with <em>multiViewMethod</em> argument. In this example, suppose the
|
|
|
243
|
+first 10 variables in the asthma data set are from a certain source and
|
|
|
244
|
+the remaining 1990 variables are from a second source. To integrate
|
|
|
245
|
+multiple data sets, each variable must be labeled with the data set it
|
|
|
246
|
+came from. This is done in a different manner depending on the data type
|
|
|
247
|
+of <em>measurements</em>.</p>
|
|
|
248
|
+<p>If using Bioconductor’s <em>DataFrame</em>, this can be specified
|
|
|
249
|
+using <em>mcols</em>. In the column metadata, each feature must have an
|
|
|
250
|
+<em>assay</em> and a <em>feature</em> name.</p>
|
|
|
251
|
+<div class="sourceCode" id="cb12"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a>measurementsDF <span class="ot"><-</span> <span class="fu">DataFrame</span>(measurements)</span>
|
|
|
252
|
+<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a><span class="fu">mcols</span>(measurementsDF) <span class="ot"><-</span> <span class="fu">data.frame</span>(</span>
|
|
|
253
|
+<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a> <span class="at">assay =</span> <span class="fu">rep</span>(<span class="fu">c</span>(<span class="st">"assay_1"</span>, <span class="st">"assay_2"</span>), <span class="at">times =</span> <span class="fu">c</span>(<span class="dv">10</span>, <span class="dv">1990</span>)),</span>
|
|
|
254
|
+<span id="cb12-4"><a href="#cb12-4" aria-hidden="true" tabindex="-1"></a> <span class="at">feature =</span> <span class="fu">colnames</span>(measurementsDF)</span>
|
|
|
255
|
+<span id="cb12-5"><a href="#cb12-5" aria-hidden="true" tabindex="-1"></a>)</span>
|
|
|
256
|
+<span id="cb12-6"><a href="#cb12-6" aria-hidden="true" tabindex="-1"></a></span>
|
|
|
257
|
+<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a>result <span class="ot"><-</span> <span class="fu">crossValidate</span>(measurementsDF, classes, <span class="at">classifier =</span> <span class="st">"SVM"</span>, <span class="at">nFolds =</span> <span class="dv">5</span>,</span>
|
|
|
258
|
+<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a> <span class="at">nRepeats =</span> <span class="dv">3</span>, <span class="at">multiViewMethod =</span> <span class="st">"merge"</span>)</span></code></pre></div>
|
|
|
259
|
+<pre><code>## Processing sample set 10.
|
|
|
260
|
+## Processing sample set 10.
|
|
|
261
|
+## Processing sample set 10.</code></pre>
|
|
|
262
|
+<div class="sourceCode" id="cb14"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="fu">performancePlot</span>(result, <span class="at">characteristicsList =</span> <span class="fu">list</span>(<span class="at">x =</span> <span class="st">"Assay Name"</span>))</span></code></pre></div>
|
|
|
263
|
+<pre><code>## Warning in .local(results, ...): Balanced Accuracy not found in all elements of results. Calculating it now.</code></pre>
|
|
|
264
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-6-1.png" width="700" /></p>
|
|
|
265
|
+<p>If using a list of <em>data.frame</em>s, the name of each element in
|
|
|
266
|
+the list will be used as the assay name.</p>
|
|
|
267
|
+<div class="sourceCode" id="cb16"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Assigns first 10 variables to dataset_1, and the rest to dataset_2</span></span>
|
|
|
268
|
+<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a>measurementsList <span class="ot"><-</span> <span class="fu">list</span>(</span>
|
|
|
269
|
+<span id="cb16-3"><a href="#cb16-3" aria-hidden="true" tabindex="-1"></a> (measurements <span class="sc">|></span> <span class="fu">as.data.frame</span>())[<span class="dv">1</span><span class="sc">:</span><span class="dv">10</span>],</span>
|
|
|
270
|
+<span id="cb16-4"><a href="#cb16-4" aria-hidden="true" tabindex="-1"></a> (measurements <span class="sc">|></span> <span class="fu">as.data.frame</span>())[<span class="dv">11</span><span class="sc">:</span><span class="dv">2000</span>]</span>
|
|
|
271
|
+<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a>)</span>
|
|
|
272
|
+<span id="cb16-6"><a href="#cb16-6" aria-hidden="true" tabindex="-1"></a><span class="fu">names</span>(measurementsList) <span class="ot"><-</span> <span class="fu">c</span>(<span class="st">"assay_1"</span>, <span class="st">"assay_2"</span>)</span>
|
|
|
273
|
+<span id="cb16-7"><a href="#cb16-7" aria-hidden="true" tabindex="-1"></a></span>
|
|
|
274
|
+<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a>result <span class="ot"><-</span> <span class="fu">crossValidate</span>(measurementsList, classes, <span class="at">classifier =</span> <span class="st">"SVM"</span>, <span class="at">nFolds =</span> <span class="dv">5</span>,</span>
|
|
|
275
|
+<span id="cb16-9"><a href="#cb16-9" aria-hidden="true" tabindex="-1"></a> <span class="at">nRepeats =</span> <span class="dv">3</span>, <span class="at">multiViewMethod =</span> <span class="st">"merge"</span>)</span></code></pre></div>
|
|
|
276
|
+<pre><code>## Processing sample set 10.
|
|
|
277
|
+## Processing sample set 10.
|
|
|
278
|
+## Processing sample set 10.</code></pre>
|
|
|
279
|
+<div class="sourceCode" id="cb18"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb18-1"><a href="#cb18-1" aria-hidden="true" tabindex="-1"></a><span class="fu">performancePlot</span>(result, <span class="at">characteristicsList =</span> <span class="fu">list</span>(<span class="at">x =</span> <span class="st">"Assay Name"</span>))</span></code></pre></div>
|
|
|
280
|
+<pre><code>## Warning in .local(results, ...): Balanced Accuracy not found in all elements of results. Calculating it now.</code></pre>
|
|
|
281
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-7-1.png" width="700" /></p>
|
|
|
282
|
+</div>
|
|
|
283
|
+</div>
|
|
|
284
|
+<div id="a-more-detailed-look-at-classifyr" class="section level2">
|
|
|
285
|
+<h2>A More Detailed Look at ClassifyR</h2>
|
|
|
286
|
+<p>In the following sections, some of the most useful functions provided
|
|
|
287
|
+in <strong>ClassifyR</strong> will be demonstrated. However, a user
|
|
|
288
|
+could wrap any feature selection, training, or prediction function to
|
|
|
289
|
+the classification framework, as long as it meets some simple rules
|
|
|
290
|
+about the input and return parameters. See the appendix section of this
|
|
|
291
|
+guide titled “Rules for New Functions” for a description of these.</p>
|
|
|
292
|
+<div id="comparison-to-existing-classification-frameworks"
|
|
|
293
|
+class="section level3">
|
|
|
294
|
+<h3>Comparison to Existing Classification Frameworks</h3>
|
|
|
295
|
+<p>There are a few other frameworks for classification in R. The table
|
|
|
296
|
+below provides a comparison of which features they offer.</p>
|
|
|
297
|
+<table>
|
|
|
298
|
+<colgroup>
|
|
|
299
|
+<col width="8%" />
|
|
|
300
|
+<col width="10%" />
|
|
|
301
|
+<col width="8%" />
|
|
|
302
|
+<col width="10%" />
|
|
|
303
|
+<col width="10%" />
|
|
|
304
|
+<col width="11%" />
|
|
|
305
|
+<col width="14%" />
|
|
|
306
|
+<col width="12%" />
|
|
|
307
|
+<col width="12%" />
|
|
|
308
|
+</colgroup>
|
|
|
309
|
+<thead>
|
|
|
310
|
+<tr class="header">
|
|
|
311
|
+<th>Package</th>
|
|
|
312
|
+<th>Run User-defined Classifiers</th>
|
|
|
313
|
+<th>Parallel Execution on any OS</th>
|
|
|
314
|
+<th>Parameter Tuning</th>
|
|
|
315
|
+<th>Intel DAAL Performance Metrics</th>
|
|
|
316
|
+<th>Ranking and Selection Plots</th>
|
|
|
317
|
+<th>Class Distribution Plot</th>
|
|
|
318
|
+<th>Sample-wise Error Heatmap</th>
|
|
|
319
|
+<th>Direct Support for MultiAssayExperiment Input</th>
|
|
|
320
|
+</tr>
|
|
|
321
|
+</thead>
|
|
|
322
|
+<tbody>
|
|
|
323
|
+<tr class="odd">
|
|
|
324
|
+<td><strong>ClassifyR</strong></td>
|
|
|
325
|
+<td>Yes</td>
|
|
|
326
|
+<td>Yes</td>
|
|
|
327
|
+<td>Yes</td>
|
|
|
328
|
+<td>Yes</td>
|
|
|
329
|
+<td>Yes</td>
|
|
|
330
|
+<td>Yes</td>
|
|
|
331
|
+<td>Yes</td>
|
|
|
332
|
+<td>Yes</td>
|
|
|
333
|
+</tr>
|
|
|
334
|
+<tr class="even">
|
|
|
335
|
+<td>caret</td>
|
|
|
336
|
+<td>Yes</td>
|
|
|
337
|
+<td>Yes</td>
|
|
|
338
|
+<td>Yes</td>
|
|
|
339
|
+<td>No</td>
|
|
|
340
|
+<td>No</td>
|
|
|
341
|
+<td>No</td>
|
|
|
342
|
+<td>No</td>
|
|
|
343
|
+<td>No</td>
|
|
|
344
|
+</tr>
|
|
|
345
|
+<tr class="odd">
|
|
|
346
|
+<td>MLInterfaces</td>
|
|
|
347
|
+<td>Yes</td>
|
|
|
348
|
+<td>No</td>
|
|
|
349
|
+<td>No</td>
|
|
|
350
|
+<td>No</td>
|
|
|
351
|
+<td>No</td>
|
|
|
352
|
+<td>No</td>
|
|
|
353
|
+<td>No</td>
|
|
|
354
|
+<td>No</td>
|
|
|
355
|
+</tr>
|
|
|
356
|
+<tr class="even">
|
|
|
357
|
+<td>MCRestimate</td>
|
|
|
358
|
+<td>Yes</td>
|
|
|
359
|
+<td>No</td>
|
|
|
360
|
+<td>Yes</td>
|
|
|
361
|
+<td>No</td>
|
|
|
362
|
+<td>No</td>
|
|
|
363
|
+<td>No</td>
|
|
|
364
|
+<td>No</td>
|
|
|
365
|
+<td>No</td>
|
|
|
366
|
+</tr>
|
|
|
367
|
+<tr class="odd">
|
|
|
368
|
+<td>CMA</td>
|
|
|
369
|
+<td>No</td>
|
|
|
370
|
+<td>No</td>
|
|
|
371
|
+<td>Yes</td>
|
|
|
372
|
+<td>No</td>
|
|
|
373
|
+<td>No</td>
|
|
|
374
|
+<td>No</td>
|
|
|
375
|
+<td>No</td>
|
|
|
376
|
+<td>No</td>
|
|
|
377
|
+</tr>
|
|
|
378
|
+</tbody>
|
|
|
379
|
+</table>
|
|
|
380
|
+</div>
|
|
|
381
|
+<div id="provided-functionality" class="section level3">
|
|
|
382
|
+<h3>Provided Functionality</h3>
|
|
|
383
|
+<p>Although being a cross-validation framework, a number of popular
|
|
|
384
|
+feature selection and classification functions are provided by the
|
|
|
385
|
+package which meet the requirements of functions to be used by it (see
|
|
|
386
|
+the last section).</p>
|
|
|
387
|
+<div id="provided-methods-for-feature-selection-and-classification"
|
|
|
388
|
+class="section level4">
|
|
|
389
|
+<h4>Provided Methods for Feature Selection and Classification</h4>
|
|
|
390
|
+<p>In the following tables, a function that is used when no function is
|
|
|
391
|
+explicitly specified by the user is shown as <span
|
|
|
392
|
+style="padding:4px; border:2px dashed #e64626;">functionName</span>.</p>
|
|
|
393
|
+<p>The functions below produce a ranking, of which different size
|
|
|
394
|
+subsets are tried and the classifier performance evaluated, to select a
|
|
|
395
|
+best subset of features, based on a criterion such as balanced accuracy
|
|
|
396
|
+rate, for example.</p>
|
|
|
397
|
+<table style="width:100%;">
|
|
|
398
|
+<colgroup>
|
|
|
399
|
+<col width="9%" />
|
|
|
400
|
+<col width="62%" />
|
|
|
401
|
+<col width="9%" />
|
|
|
402
|
+<col width="9%" />
|
|
|
403
|
+<col width="9%" />
|
|
|
404
|
+</colgroup>
|
|
|
405
|
+<thead>
|
|
|
406
|
+<tr class="header">
|
|
|
407
|
+<th>Function</th>
|
|
|
408
|
+<th>Description</th>
|
|
|
409
|
+<th>DM</th>
|
|
|
410
|
+<th>DV</th>
|
|
|
411
|
+<th>DD</th>
|
|
|
412
|
+</tr>
|
|
|
413
|
+</thead>
|
|
|
414
|
+<tbody>
|
|
|
415
|
+<tr class="odd">
|
|
|
416
|
+<td><span
|
|
|
417
|
+style="padding:4px; border:2px dashed #e64626; font-family: 'Courier New', monospace;">differentMeansRanking</span></td>
|
|
|
418
|
+<td>t-test ranking if two classes, F-test ranking if three or more</td>
|
|
|
419
|
+<td>✔</td>
|
|
|
420
|
+<td></td>
|
|
|
421
|
+<td></td>
|
|
|
422
|
+</tr>
|
|
|
423
|
+<tr class="even">
|
|
|
424
|
+<td><span
|
|
|
425
|
+style="font-family: 'Courier New', monospace;">limmaRanking</span></td>
|
|
|
426
|
+<td>Moderated t-test ranking using variance shrinkage</td>
|
|
|
427
|
+<td>✔</td>
|
|
|
428
|
+<td></td>
|
|
|
429
|
+<td></td>
|
|
|
430
|
+</tr>
|
|
|
431
|
+<tr class="odd">
|
|
|
432
|
+<td><span
|
|
|
433
|
+style="font-family: 'Courier New', monospace;">edgeRranking</span></td>
|
|
|
434
|
+<td>Likelihood ratio test for count data ranking</td>
|
|
|
435
|
+<td>✔</td>
|
|
|
436
|
+<td></td>
|
|
|
437
|
+<td></td>
|
|
|
438
|
+</tr>
|
|
|
439
|
+<tr class="even">
|
|
|
440
|
+<td><span
|
|
|
441
|
+style="font-family: 'Courier New', monospace;">bartlettRanking</span></td>
|
|
|
442
|
+<td>Bartlett’s test non-robust ranking</td>
|
|
|
443
|
+<td></td>
|
|
|
444
|
+<td>✔</td>
|
|
|
445
|
+<td></td>
|
|
|
446
|
+</tr>
|
|
|
447
|
+<tr class="odd">
|
|
|
448
|
+<td><span
|
|
|
449
|
+style="font-family: 'Courier New', monospace;">leveneRanking</span></td>
|
|
|
450
|
+<td>Levene’s test robust ranking</td>
|
|
|
451
|
+<td></td>
|
|
|
452
|
+<td>✔</td>
|
|
|
453
|
+<td></td>
|
|
|
454
|
+</tr>
|
|
|
455
|
+<tr class="even">
|
|
|
456
|
+<td><span
|
|
|
457
|
+style="font-family: 'Courier New', monospace;">DMDranking</span></td>
|
|
|
458
|
+<td><span style="white-space: nowrap">Difference in location
|
|
|
459
|
+(mean/median) and/or scale (SD, MAD, <span
|
|
|
460
|
+class="math inline">\(Q_n\)</span>)</span></td>
|
|
|
461
|
+<td>✔</td>
|
|
|
462
|
+<td>✔</td>
|
|
|
463
|
+<td>✔</td>
|
|
|
464
|
+</tr>
|
|
|
465
|
+<tr class="odd">
|
|
|
466
|
+<td><span
|
|
|
467
|
+style="font-family: 'Courier New', monospace;">likelihoodRatioRanking</span></td>
|
|
|
468
|
+<td>Likelihood ratio (normal distribution) ranking</td>
|
|
|
469
|
+<td>✔</td>
|
|
|
470
|
+<td>✔</td>
|
|
|
471
|
+<td>✔</td>
|
|
|
472
|
+</tr>
|
|
|
473
|
+<tr class="even">
|
|
|
474
|
+<td><span
|
|
|
475
|
+style="font-family: 'Courier New', monospace;">KolmogorovSmirnovRanking</span></td>
|
|
|
476
|
+<td>Kolmogorov-Smirnov distance between distributions ranking</td>
|
|
|
477
|
+<td>✔</td>
|
|
|
478
|
+<td>✔</td>
|
|
|
479
|
+<td>✔</td>
|
|
|
480
|
+</tr>
|
|
|
481
|
+<tr class="odd">
|
|
|
482
|
+<td><span
|
|
|
483
|
+style="font-family: 'Courier New', monospace;">KullbackLeiblerRanking</span></td>
|
|
|
484
|
+<td>Kullback-Leibler distance between distributions ranking</td>
|
|
|
485
|
+<td>✔</td>
|
|
|
486
|
+<td>✔</td>
|
|
|
487
|
+<td>✔</td>
|
|
|
488
|
+</tr>
|
|
|
489
|
+</tbody>
|
|
|
490
|
+</table>
|
|
|
491
|
+<p>Likewise, a variety of classifiers is also provided.</p>
|
|
|
492
|
+<table>
|
|
|
493
|
+<colgroup>
|
|
|
494
|
+<col width="9%" />
|
|
|
495
|
+<col width="61%" />
|
|
|
496
|
+<col width="9%" />
|
|
|
497
|
+<col width="9%" />
|
|
|
498
|
+<col width="9%" />
|
|
|
499
|
+</colgroup>
|
|
|
500
|
+<thead>
|
|
|
501
|
+<tr class="header">
|
|
|
502
|
+<th>Function(s)</th>
|
|
|
503
|
+<th>Description</th>
|
|
|
504
|
+<th>DM</th>
|
|
|
505
|
+<th>DV</th>
|
|
|
506
|
+<th>DD</th>
|
|
|
507
|
+</tr>
|
|
|
508
|
+</thead>
|
|
|
509
|
+<tbody>
|
|
|
510
|
+<tr class="odd">
|
|
|
511
|
+<td><span
|
|
|
512
|
+style="padding:1px; border:2px dashed #e64626; display:inline-block; margin-bottom: 3px; font-family: 'Courier New', monospace;">DLDAtrainInterface</span>,<br><span
|
|
|
513
|
+style="padding:1px; border:2px dashed #e64626; display:inline-block; font-family: 'Courier New', monospace;">DLDApredictInterface</span></td>
|
|
|
514
|
+<td>Wrappers for sparsediscrim’s functions <span
|
|
|
515
|
+style="font-family: 'Courier New', monospace;">dlda</span> and
|
|
|
516
|
+<span
|
|
|
517
|
+style="font-family: 'Courier New', monospace;">predict.dlda</span>
|
|
|
518
|
+functions</td>
|
|
|
519
|
+<td>✔</td>
|
|
|
520
|
+<td></td>
|
|
|
521
|
+<td></td>
|
|
|
522
|
+</tr>
|
|
|
523
|
+<tr class="even">
|
|
|
524
|
+<td><span
|
|
|
525
|
+style="font-family: 'Courier New', monospace;">classifyInterface</span></td>
|
|
|
526
|
+<td>Wrapper for PoiClaClu’s Poisson LDA function <span
|
|
|
527
|
+style="font-family: 'Courier New', monospace;">classify</span></td>
|
|
|
528
|
+<td>✔</td>
|
|
|
529
|
+<td></td>
|
|
|
530
|
+<td></td>
|
|
|
531
|
+</tr>
|
|
|
532
|
+<tr class="odd">
|
|
|
533
|
+<td><span
|
|
|
534
|
+style="font-family: 'Courier New', monospace;">elasticNetGLMtrainInterface</span>,
|
|
|
535
|
+<span
|
|
|
536
|
+style="font-family: 'Courier New', monospace;">elasticNetGLMpredictInterface</span></td>
|
|
|
537
|
+<td>Wrappers for glmnet’s elastic net GLM functions <span
|
|
|
538
|
+style="font-family: 'Courier New', monospace;">glmnet</span> and
|
|
|
539
|
+<span
|
|
|
540
|
+style="font-family: 'Courier New', monospace;">predict.glmnet</span></td>
|
|
|
541
|
+<td>✔</td>
|
|
|
542
|
+<td></td>
|
|
|
543
|
+<td></td>
|
|
|
544
|
+</tr>
|
|
|
545
|
+<tr class="even">
|
|
|
546
|
+<td><span
|
|
|
547
|
+style="font-family: 'Courier New', monospace;">NSCtrainInterface</span>,
|
|
|
548
|
+<span
|
|
|
549
|
+style="font-family: 'Courier New', monospace;">NSCpredictInterface</span></td>
|
|
|
550
|
+<td>Wrappers for pamr’s Nearest Shrunken Centroid functions <span
|
|
|
551
|
+style="font-family: 'Courier New', monospace;">pamr.train</span>
|
|
|
552
|
+and <span
|
|
|
553
|
+style="font-family: 'Courier New', monospace;">pamr.predict</span></td>
|
|
|
554
|
+<td>✔</td>
|
|
|
555
|
+<td></td>
|
|
|
556
|
+<td></td>
|
|
|
557
|
+</tr>
|
|
|
558
|
+<tr class="odd">
|
|
|
559
|
+<td><span
|
|
|
560
|
+style="font-family: 'Courier New', monospace;">fisherDiscriminant</span></td>
|
|
|
561
|
+<td>Implementation of Fisher’s LDA for departures from normality</td>
|
|
|
562
|
+<td>✔</td>
|
|
|
563
|
+<td>✔*</td>
|
|
|
564
|
+<td></td>
|
|
|
565
|
+</tr>
|
|
|
566
|
+<tr class="even">
|
|
|
567
|
+<td><span
|
|
|
568
|
+style="font-family: 'Courier New', monospace;">mixModelsTrain</span>,
|
|
|
569
|
+<span
|
|
|
570
|
+style="font-family: 'Courier New', monospace;">mixModelsPredict</span></td>
|
|
|
571
|
+<td>Feature-wise mixtures of normals and voting</td>
|
|
|
572
|
+<td>✔</td>
|
|
|
573
|
+<td>✔</td>
|
|
|
574
|
+<td>✔</td>
|
|
|
575
|
+</tr>
|
|
|
576
|
+<tr class="odd">
|
|
|
577
|
+<td><span
|
|
|
578
|
+style="font-family: 'Courier New', monospace;">naiveBayesKernel</span></td>
|
|
|
579
|
+<td>Feature-wise kernel density estimation and voting</td>
|
|
|
580
|
+<td>✔</td>
|
|
|
581
|
+<td>✔</td>
|
|
|
582
|
+<td>✔</td>
|
|
|
583
|
+</tr>
|
|
|
584
|
+<tr class="even">
|
|
|
585
|
+<td><span
|
|
|
586
|
+style="font-family: 'Courier New', monospace;">randomForestTrainInterface</span>,
|
|
|
587
|
+<span
|
|
|
588
|
+style="font-family: 'Courier New', monospace;">randomForestPredictInterface</span></td>
|
|
|
589
|
+<td>Wrapper for ranger’s functions <span
|
|
|
590
|
+style="font-family: 'Courier New', monospace;">ranger</span> and
|
|
|
591
|
+<span
|
|
|
592
|
+style="font-family: 'Courier New', monospace;">predict</span></td>
|
|
|
593
|
+<td>✔</td>
|
|
|
594
|
+<td>✔</td>
|
|
|
595
|
+<td>✔</td>
|
|
|
596
|
+</tr>
|
|
|
597
|
+<tr class="odd">
|
|
|
598
|
+<td><span
|
|
|
599
|
+style="font-family: 'Courier New', monospace;">extremeGradientBoostingTrainInterface</span>,
|
|
|
600
|
+<span
|
|
|
601
|
+style="font-family: 'Courier New', monospace;">extremeGradientBoostingPredictInterface</span></td>
|
|
|
602
|
+<td>Wrapper for xgboost’s functions <span
|
|
|
603
|
+style="font-family: 'Courier New', monospace;">xgboost</span>
|
|
|
604
|
+and <span
|
|
|
605
|
+style="font-family: 'Courier New', monospace;">predict</span></td>
|
|
|
606
|
+<td>✔</td>
|
|
|
607
|
+<td>✔</td>
|
|
|
608
|
+<td>✔</td>
|
|
|
609
|
+</tr>
|
|
|
610
|
+<tr class="even">
|
|
|
611
|
+<td><span
|
|
|
612
|
+style="font-family: 'Courier New', monospace;">kNNinterface</span></td>
|
|
|
613
|
+<td>Wrapper for class’s function <span
|
|
|
614
|
+style="font-family: 'Courier New', monospace;">knn</span></td>
|
|
|
615
|
+<td>✔</td>
|
|
|
616
|
+<td>✔</td>
|
|
|
617
|
+<td>✔</td>
|
|
|
618
|
+</tr>
|
|
|
619
|
+<tr class="odd">
|
|
|
620
|
+<td><span
|
|
|
621
|
+style="font-family: 'Courier New', monospace;">SVMtrainInterface</span>,
|
|
|
622
|
+<span
|
|
|
623
|
+style="font-family: 'Courier New', monospace;">SVMpredictInterface</span></td>
|
|
|
624
|
+<td>Wrapper for e1071’s functions <span
|
|
|
625
|
+style="font-family: 'Courier New', monospace;">svm</span> and
|
|
|
626
|
+<span
|
|
|
627
|
+style="font-family: 'Courier New', monospace;">predict.svm</span></td>
|
|
|
628
|
+<td>✔</td>
|
|
|
629
|
+<td>✔ †</td>
|
|
|
630
|
+<td>✔ †</td>
|
|
|
631
|
+</tr>
|
|
|
632
|
+</tbody>
|
|
|
633
|
+</table>
|
|
|
634
|
+<p>* If ordinary numeric measurements have been transformed to absolute
|
|
|
635
|
+deviations using <span
|
|
|
636
|
+style="font-family: 'Courier New', monospace;">subtractFromLocation</span>.<br>
|
|
|
637
|
+† If the value of <span
|
|
|
638
|
+style="font-family: 'Courier New', monospace;">kernel</span> is
|
|
|
639
|
+not <span
|
|
|
640
|
+style="font-family: 'Courier New', monospace;">“linear”</span>.</p>
|
|
|
641
|
+<p>If a desired selection or classification method is not already
|
|
|
642
|
+implemented, rules for writing functions to work with
|
|
|
643
|
+<strong>ClassifyR</strong> are outlined in the wrapper vignette. Please
|
|
|
644
|
+visit it for more information.</p>
|
|
|
645
|
+</div>
|
|
|
646
|
+<div id="provided-meta-feature-methods" class="section level4">
|
|
|
647
|
+<h4>Provided Meta-feature Methods</h4>
|
|
|
648
|
+<p>A number of methods are provided for users to enable classification
|
|
|
649
|
+in a feature-set-centric or interactor-centric way. The meta-feature
|
|
|
650
|
+creation functions should be used before cross-validation is done.</p>
|
|
|
651
|
+<table>
|
|
|
652
|
+<colgroup>
|
|
|
653
|
+<col width="9%" />
|
|
|
654
|
+<col width="61%" />
|
|
|
655
|
+<col width="14%" />
|
|
|
656
|
+<col width="14%" />
|
|
|
657
|
+</colgroup>
|
|
|
658
|
+<thead>
|
|
|
659
|
+<tr class="header">
|
|
|
660
|
+<th>Function</th>
|
|
|
661
|
+<th>Description</th>
|
|
|
662
|
+<th align="center">Before CV</th>
|
|
|
663
|
+<th align="center">During CV</th>
|
|
|
664
|
+</tr>
|
|
|
665
|
+</thead>
|
|
|
666
|
+<tbody>
|
|
|
667
|
+<tr class="odd">
|
|
|
668
|
+<td><span
|
|
|
669
|
+style="font-family: 'Courier New', monospace;">edgesToHubNetworks</span></td>
|
|
|
670
|
+<td>Takes a two-column <span
|
|
|
671
|
+style="font-family: 'Courier New', monospace;">matrix</span> or
|
|
|
672
|
+<span
|
|
|
673
|
+style="font-family: 'Courier New', monospace;">DataFrame</span>
|
|
|
674
|
+and finds all nodes with at least a minimum number of interactions</td>
|
|
|
675
|
+<td align="center">✔</td>
|
|
|
676
|
+<td align="center"></td>
|
|
|
677
|
+</tr>
|
|
|
678
|
+<tr class="even">
|
|
|
679
|
+<td><span
|
|
|
680
|
+style="font-family: 'Courier New', monospace;">featureSetSummary</span></td>
|
|
|
681
|
+<td><span style="white-space: nowrap">Considers sets of features and
|
|
|
682
|
+calculates their mean or median</span></td>
|
|
|
683
|
+<td align="center">✔</td>
|
|
|
684
|
+<td align="center"></td>
|
|
|
685
|
+</tr>
|
|
|
686
|
+<tr class="odd">
|
|
|
687
|
+<td><span
|
|
|
688
|
+style="font-family: 'Courier New', monospace;">pairsDifferencesSelection</span></td>
|
|
|
689
|
+<td>Finds a set of pairs of features whose measurement inequalities can
|
|
|
690
|
+be used for predicting with</td>
|
|
|
691
|
+<td align="center"></td>
|
|
|
692
|
+<td align="center">✔</td>
|
|
|
693
|
+</tr>
|
|
|
694
|
+<tr class="even">
|
|
|
695
|
+<td><span
|
|
|
696
|
+style="font-family: 'Courier New', monospace;">kTSPclassifier</span></td>
|
|
|
697
|
+<td>Voting classifier that uses inequalities between pairs of features
|
|
|
698
|
+to vote for one of two classes</td>
|
|
|
699
|
+<td align="center"></td>
|
|
|
700
|
+<td align="center">✔</td>
|
|
|
701
|
+</tr>
|
|
|
702
|
+</tbody>
|
|
|
703
|
+</table>
|
|
|
704
|
+</div>
|
|
|
705
|
+</div>
|
|
|
706
|
+<div id="fine-grained-cross-validation-and-modelling-using-runtests"
|
|
|
707
|
+class="section level3">
|
|
|
708
|
+<h3>Fine-grained Cross-validation and Modelling Using
|
|
|
709
|
+<em>runTests</em></h3>
|
|
|
710
|
+<p>For more control over the finer aspects of cross-validation of a
|
|
|
711
|
+single data set, <em>runTests</em> may be employed in place of
|
|
|
712
|
+<em>crossValidate</em>. For the variety of cross-validation, the
|
|
|
713
|
+parameters are specified by a <em>CrossValParams</em> object. The
|
|
|
714
|
+default setting is for 100 permutations and five folds and parameter
|
|
|
715
|
+tuning is done by resubstitution. It is also recommended to specify a
|
|
|
716
|
+<em>parallelParams</em> setting. On Linux and MacOS operating systems,
|
|
|
717
|
+it should be <em>MulticoreParam</em> and on Windows computers it should
|
|
|
718
|
+be <em>SnowParam</em>. Note that each of these have an option
|
|
|
719
|
+<em>RNGseed</em> and this <strong>needs to be set by the user</strong>
|
|
|
720
|
+because some classifiers or feature selection functions will have some
|
|
|
721
|
+element of randomisation. One example that works on all operating
|
|
|
722
|
+systems, but is best-suited to Windows is:</p>
|
|
|
723
|
+<div class="sourceCode" id="cb20"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a>CVparams <span class="ot"><-</span> <span class="fu">CrossValParams</span>(<span class="at">parallelParams =</span> <span class="fu">SnowParam</span>(<span class="dv">16</span>, <span class="at">RNGseed =</span> <span class="dv">123</span>))</span>
|
|
|
724
|
+<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a>CVparams</span></code></pre></div>
|
|
|
725
|
+<p>For the actual operations to do to the data to build a model of it,
|
|
|
726
|
+each of the stages should be specified by an object of class
|
|
|
727
|
+<em>ModellingParams</em>. This controls how class imbalance is handled
|
|
|
728
|
+(default is to downsample to the smallest class), any transformation
|
|
|
729
|
+that needs to be done inside of cross-validation (i.e. involving a
|
|
|
730
|
+computed value from the training set), any feature selection and the
|
|
|
731
|
+training and prediction functions to be used. The default is to do an
|
|
|
732
|
+ordinary t-test (two groups) or ANOVA (three or more groups) and
|
|
|
733
|
+classification using diagonal LDA.</p>
|
|
|
734
|
+<div class="sourceCode" id="cb21"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ModellingParams</span>()</span></code></pre></div>
|
|
|
735
|
+<pre><code>## An object of class "ModellingParams"
|
|
|
736
|
+## Slot "balancing":
|
|
|
737
|
+## [1] "downsample"
|
|
|
738
|
+##
|
|
|
739
|
+## Slot "transformParams":
|
|
|
740
|
+## NULL
|
|
|
741
|
+##
|
|
|
742
|
+## Slot "selectParams":
|
|
|
743
|
+## An object of class 'SelectParams'.
|
|
|
744
|
+## Selection Name: Difference in Means.
|
|
|
745
|
+##
|
|
|
746
|
+## Slot "trainParams":
|
|
|
747
|
+## An object of class 'TrainParams'.
|
|
|
748
|
+## Classifier Name: Diagonal LDA.
|
|
|
749
|
+##
|
|
|
750
|
+## Slot "predictParams":
|
|
|
751
|
+## An object of class 'PredictParams'.
|
|
|
752
|
+##
|
|
|
753
|
+## Slot "doImportance":
|
|
|
754
|
+## [1] FALSE</code></pre>
|
|
|
755
|
+</div>
|
|
|
756
|
+<div id="runtests-driver-function-of-cross-validated-classification"
|
|
|
757
|
+class="section level3">
|
|
|
758
|
+<h3>runTests Driver Function of Cross-validated Classification</h3>
|
|
|
759
|
+<p><em>runTests</em> is the main function in <strong>ClassifyR</strong>
|
|
|
760
|
+which handles the sample splitting and parallelisation, if used, of
|
|
|
761
|
+cross-validation. To begin with, a simple classifier will be
|
|
|
762
|
+demonstrated. It uses a t-test or ANOVA ranking (depending on the number
|
|
|
763
|
+of classes) for feature ranking and DLDA for classification. This
|
|
|
764
|
+classifier relies on differences in means between classes. No parameters
|
|
|
765
|
+need to be specified, because this is the default classification of
|
|
|
766
|
+<em>runTests</em>. By default, the number of features is tuned by
|
|
|
767
|
+resubstitution on the training set.</p>
|
|
|
768
|
+<div class="sourceCode" id="cb23"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a>crossValParams <span class="ot"><-</span> <span class="fu">CrossValParams</span>(<span class="at">permutations =</span> <span class="dv">5</span>)</span>
|
|
|
769
|
+<span id="cb23-2"><a href="#cb23-2" aria-hidden="true" tabindex="-1"></a>DMresults <span class="ot"><-</span> <span class="fu">runTests</span>(measurements, classes, crossValParams, <span class="at">verbose =</span> <span class="dv">1</span>)</span></code></pre></div>
|
|
|
770
|
+<pre><code>## Processing sample set 10.</code></pre>
|
|
|
771
|
+<pre><code>## Processing sample set 20.</code></pre>
|
|
|
772
|
+<p>Here, 5 permutations (non-default) and 5 folds cross-validation
|
|
|
773
|
+(default) is specified. For computers with more than 1 CPU, the number
|
|
|
774
|
+of cores to use can be given to <em>runTests</em> by using the argument
|
|
|
775
|
+<em>parallelParams</em>. The parameter <em>seed</em> is important to set
|
|
|
776
|
+for result reproducibility when doing a cross-validation such as this,
|
|
|
777
|
+because it employs randomisation to partition the samples into folds.
|
|
|
778
|
+Also, <em>RNGseed</em> is highly recommended to be set to the back-end
|
|
|
779
|
+specified to <em>BPPARAM</em> if doing parallel processing. The first
|
|
|
780
|
+seed mentioned does not work for parallel processes. For more details
|
|
|
781
|
+about <em>runTests</em> and the parameter classes used by it, consult
|
|
|
782
|
+the help pages of such functions.</p>
|
|
|
783
|
+</div>
|
|
|
784
|
+</div>
|
|
|
785
|
+<div id="evaluation-of-a-classification" class="section level2">
|
|
|
786
|
+<h2>Evaluation of a Classification</h2>
|
|
|
787
|
+<p>The most frequently selected gene can be identified using the
|
|
|
788
|
+<em>distribution</em> function and its relative abundance values for all
|
|
|
789
|
+samples can be displayed visually by <em>plotFeatureClasses</em>.</p>
|
|
|
790
|
+<div class="sourceCode" id="cb26"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb26-1"><a href="#cb26-1" aria-hidden="true" tabindex="-1"></a>selectionPercentages <span class="ot"><-</span> <span class="fu">distribution</span>(DMresults, <span class="at">plot =</span> <span class="cn">FALSE</span>)</span>
|
|
|
791
|
+<span id="cb26-2"><a href="#cb26-2" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(selectionPercentages)</span>
|
|
|
792
|
+<span id="cb26-3"><a href="#cb26-3" aria-hidden="true" tabindex="-1"></a>sortedPercentages <span class="ot"><-</span> <span class="fu">head</span>(selectionPercentages[<span class="fu">order</span>(selectionPercentages, <span class="at">decreasing =</span> <span class="cn">TRUE</span>)])</span>
|
|
|
793
|
+<span id="cb26-4"><a href="#cb26-4" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span>(sortedPercentages)</span>
|
|
|
794
|
+<span id="cb26-5"><a href="#cb26-5" aria-hidden="true" tabindex="-1"></a>mostChosen <span class="ot"><-</span> sortedPercentages[<span class="dv">1</span>]</span>
|
|
|
795
|
+<span id="cb26-6"><a href="#cb26-6" aria-hidden="true" tabindex="-1"></a>bestGenePlot <span class="ot"><-</span> <span class="fu">plotFeatureClasses</span>(measurements, classes, <span class="fu">names</span>(mostChosen), <span class="at">dotBinWidth =</span> <span class="fl">0.1</span>,</span>
|
|
|
796
|
+<span id="cb26-7"><a href="#cb26-7" aria-hidden="true" tabindex="-1"></a> <span class="at">xAxisLabel =</span> <span class="st">"Normalised Expression"</span>)</span></code></pre></div>
|
|
|
797
|
+<pre><code>## Warning: [1m[22mThe dot-dot notation (`..density..`) was deprecated in ggplot2 3.4.0.
|
|
|
798
|
+## [36mℹ[39m Please use `after_stat(density)` instead.
|
|
|
799
|
+## [36mℹ[39m The deprecated feature was likely used in the [34mClassifyR[39m package.
|
|
|
800
|
+## Please report the issue to the authors.</code></pre>
|
|
|
801
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-11-1.png" width="768" /></p>
|
|
|
802
|
+<pre><code>## allFeaturesText
|
|
|
803
|
+## ANKMY1 ARHGAP39 C10orf95 C19orf51 C2orf55 C6orf108
|
|
|
804
|
+## 8 64 100 80 4 12
|
|
|
805
|
+## allFeaturesText
|
|
|
806
|
+## C10orf95 CROCC SSBP4 ZDHHC1 TMEM190 C19orf51
|
|
|
807
|
+## 100 100 100 100 84 80</code></pre>
|
|
|
808
|
+<p>The means of the abundance levels of C10orf95 are substantially
|
|
|
809
|
+different between the people with and without asthma.
|
|
|
810
|
+<em>plotFeatureClasses</em> can also plot categorical data, such as may
|
|
|
811
|
+be found in a clinical data table, as a bar chart.</p>
|
|
|
812
|
+<p>Classification error rates, as well as many other prediction
|
|
|
813
|
+performance measures, can be calculated with <em>calcCVperformance</em>.
|
|
|
814
|
+Next, the balanced accuracy rate is calculated considering all samples,
|
|
|
815
|
+each of which was in the test set once. The balanced accuracy rate is
|
|
|
816
|
+defined as the average rate of the correct classifications of each
|
|
|
817
|
+class.</p>
|
|
|
818
|
+<p>See the documentation of <em>calcCVperformance</em> for a list of
|
|
|
819
|
+performance metrics which may be calculated.</p>
|
|
|
820
|
+<div class="sourceCode" id="cb29"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb29-1"><a href="#cb29-1" aria-hidden="true" tabindex="-1"></a>DMresults <span class="ot"><-</span> <span class="fu">calcCVperformance</span>(DMresults)</span>
|
|
|
821
|
+<span id="cb29-2"><a href="#cb29-2" aria-hidden="true" tabindex="-1"></a>DMresults</span></code></pre></div>
|
|
|
822
|
+<pre><code>## An object of class 'ClassifyResult'.
|
|
|
823
|
+## Characteristics:
|
|
|
824
|
+## characteristic value
|
|
|
825
|
+## Selection Name Difference in Means
|
|
|
826
|
+## Classifier Name Diagonal LDA
|
|
|
827
|
+## Cross-validation 5 Permutations, 5 Folds
|
|
|
828
|
+## Features: List of length 25 of feature identifiers.
|
|
|
829
|
+## Predictions: A data frame of 950 rows.
|
|
|
830
|
+## Performance Measures: Balanced Accuracy.</code></pre>
|
|
|
831
|
+<div class="sourceCode" id="cb31"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb31-1"><a href="#cb31-1" aria-hidden="true" tabindex="-1"></a><span class="fu">performance</span>(DMresults)</span></code></pre></div>
|
|
|
832
|
+<pre><code>## $`Balanced Accuracy`
|
|
|
833
|
+## 1 2 3 4 5
|
|
|
834
|
+## 0.7850684 0.7931329 0.8011975 0.8047410 0.8077957</code></pre>
|
|
|
835
|
+<p>The error rate is about 20%. If only a vector of predictions and a
|
|
|
836
|
+vector of actual classes is available, such as from an old study which
|
|
|
837
|
+did not use <strong>ClassifyR</strong> for cross-validation, then
|
|
|
838
|
+<em>calcExternalPerformance</em> can be used on a pair of factor vectors
|
|
|
839
|
+which have the same length.</p>
|
|
|
840
|
+<div id="comparison-of-different-classifications"
|
|
|
841
|
+class="section level3">
|
|
|
842
|
+<h3>Comparison of Different Classifications</h3>
|
|
|
843
|
+<p>The <em>samplesMetricMap</em> function allows the visual comparison
|
|
|
844
|
+of sample-wise error rate or accuracy measures from different
|
|
|
845
|
+<em>ClassifyResult</em> objects. Firstly, a classifier will be run that
|
|
|
846
|
+uses Kullback-Leibler divergence ranking and resubstitution error as a
|
|
|
847
|
+feature selection heuristic and a naive Bayes classifier for
|
|
|
848
|
+classification. This classification will use features that have either a
|
|
|
849
|
+change in location or in scale between classes.</p>
|
|
|
850
|
+<div class="sourceCode" id="cb33"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb33-1"><a href="#cb33-1" aria-hidden="true" tabindex="-1"></a>modellingParamsDD <span class="ot"><-</span> <span class="fu">ModellingParams</span>(<span class="at">selectParams =</span> <span class="fu">SelectParams</span>(<span class="st">"KL"</span>),</span>
|
|
|
851
|
+<span id="cb33-2"><a href="#cb33-2" aria-hidden="true" tabindex="-1"></a> <span class="at">trainParams =</span> <span class="fu">TrainParams</span>(<span class="st">"naiveBayes"</span>),</span>
|
|
|
852
|
+<span id="cb33-3"><a href="#cb33-3" aria-hidden="true" tabindex="-1"></a> <span class="at">predictParams =</span> <span class="cn">NULL</span>)</span>
|
|
|
853
|
+<span id="cb33-4"><a href="#cb33-4" aria-hidden="true" tabindex="-1"></a>DDresults <span class="ot"><-</span> <span class="fu">runTests</span>(measurements, classes, crossValParams, modellingParamsDD, <span class="at">verbose =</span> <span class="dv">1</span>)</span></code></pre></div>
|
|
|
854
|
+<pre><code>## Processing sample set 10.</code></pre>
|
|
|
855
|
+<pre><code>## Processing sample set 20.</code></pre>
|
|
|
856
|
+<div class="sourceCode" id="cb36"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb36-1"><a href="#cb36-1" aria-hidden="true" tabindex="-1"></a>DDresults</span></code></pre></div>
|
|
|
857
|
+<pre><code>## An object of class 'ClassifyResult'.
|
|
|
858
|
+## Characteristics:
|
|
|
859
|
+## characteristic value
|
|
|
860
|
+## Selection Name Kullback-Leibler Divergence
|
|
|
861
|
+## Classifier Name Naive Bayes Kernel
|
|
|
862
|
+## Cross-validation 5 Permutations, 5 Folds
|
|
|
863
|
+## Features: List of length 25 of feature identifiers.
|
|
|
864
|
+## Predictions: A data frame of 950 rows.
|
|
|
865
|
+## Performance Measures: None calculated yet.</code></pre>
|
|
|
866
|
+<p>The naive Bayes kernel classifier by default uses the vertical
|
|
|
867
|
+distance between class densities but it can instead use the horizontal
|
|
|
868
|
+distance to the nearest non-zero density cross-over point to confidently
|
|
|
869
|
+classify samples in the tails of the densities.</p>
|
|
|
870
|
+<p>Now, the classification error for each sample is also calculated for
|
|
|
871
|
+both the differential means and differential distribution classifiers
|
|
|
872
|
+and both <em>ClassifyResult</em> objects generated so far are plotted
|
|
|
873
|
+with <em>samplesMetricMap</em>.</p>
|
|
|
874
|
+<div class="sourceCode" id="cb38"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb38-1"><a href="#cb38-1" aria-hidden="true" tabindex="-1"></a>DMresults <span class="ot"><-</span> <span class="fu">calcCVperformance</span>(DMresults, <span class="st">"Sample Error"</span>)</span>
|
|
|
875
|
+<span id="cb38-2"><a href="#cb38-2" aria-hidden="true" tabindex="-1"></a>DDresults <span class="ot"><-</span> <span class="fu">calcCVperformance</span>(DDresults, <span class="st">"Sample Error"</span>)</span>
|
|
|
876
|
+<span id="cb38-3"><a href="#cb38-3" aria-hidden="true" tabindex="-1"></a>resultsList <span class="ot"><-</span> <span class="fu">list</span>(<span class="at">Abundance =</span> DMresults, <span class="at">Distribution =</span> DDresults)</span>
|
|
|
877
|
+<span id="cb38-4"><a href="#cb38-4" aria-hidden="true" tabindex="-1"></a><span class="fu">samplesMetricMap</span>(resultsList, <span class="at">metric =</span> <span class="st">"Sample Error"</span>, <span class="at">xAxisLabel =</span> <span class="st">"Sample"</span>,</span>
|
|
|
878
|
+<span id="cb38-5"><a href="#cb38-5" aria-hidden="true" tabindex="-1"></a> <span class="at">showXtickLabels =</span> <span class="cn">FALSE</span>)</span></code></pre></div>
|
|
|
879
|
+<pre><code>## Warning: [1m[22mRemoved 2 rows containing missing values (`geom_tile()`).</code></pre>
|
|
|
880
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-14-1.png" width="960" /></p>
|
|
|
881
|
+<pre><code>## TableGrob (2 x 1) "arrange": 2 grobs
|
|
|
882
|
+## z cells name grob
|
|
|
883
|
+## 1 1 (2-2,1-1) arrange gtable[layout]
|
|
|
884
|
+## 2 2 (1-1,1-1) arrange text[GRID.text.533]</code></pre>
|
|
|
885
|
+<p>The benefit of this plot is that it allows the easy identification of
|
|
|
886
|
+samples which are hard to classify and could be explained by considering
|
|
|
887
|
+additional information about them. Differential distribution class
|
|
|
888
|
+prediction appears to be biased to the majority class (No Asthma).</p>
|
|
|
889
|
+<p>More traditionally, the distribution of performance values of each
|
|
|
890
|
+complete cross-validation can be visualised by <em>performancePlot</em>
|
|
|
891
|
+by providing them as a list to the function. The default is to draw box
|
|
|
892
|
+plots, but violin plots could also be made. The default performance
|
|
|
893
|
+metric to plot is balanced accuracy. If it’s not already calculated for
|
|
|
894
|
+all classifications, as in this case for DD, it will be done
|
|
|
895
|
+automatically.</p>
|
|
|
896
|
+<div class="sourceCode" id="cb41"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb41-1"><a href="#cb41-1" aria-hidden="true" tabindex="-1"></a><span class="fu">performancePlot</span>(resultsList)</span></code></pre></div>
|
|
|
897
|
+<pre><code>## Warning in .local(results, ...): Balanced Accuracy not found in all elements of results. Calculating it now.</code></pre>
|
|
|
898
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-15-1.png" width="700" /></p>
|
|
|
899
|
+<p>We can observe that the spread of balanced accuracy rates is small,
|
|
|
900
|
+but slightly wider for the differential distribution classifier.</p>
|
|
|
901
|
+<p>The features being ranked and selected in the feature selection stage
|
|
|
902
|
+can be compared within and between classifiers by the plotting functions
|
|
|
903
|
+<em>rankingPlot</em> and <em>selectionPlot</em>. Consider the task of
|
|
|
904
|
+visually representing how consistent the feature rankings of the top 100
|
|
|
905
|
+different features were for the differential distribution classifier for
|
|
|
906
|
+all 5 folds in the 5 cross-validations.</p>
|
|
|
907
|
+<div class="sourceCode" id="cb43"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb43-1"><a href="#cb43-1" aria-hidden="true" tabindex="-1"></a><span class="fu">rankingPlot</span>(DDresults, <span class="at">topRanked =</span> <span class="dv">1</span><span class="sc">:</span><span class="dv">100</span>, <span class="at">xLabelPositions =</span> <span class="fu">c</span>(<span class="dv">1</span>, <span class="fu">seq</span>(<span class="dv">10</span>, <span class="dv">100</span>, <span class="dv">10</span>)))</span></code></pre></div>
|
|
|
908
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-16-1.png" width="700" /></p>
|
|
|
909
|
+<p>The top-ranked features are fairly similar between all pairs of the
|
|
|
910
|
+20 cross-validations.</p>
|
|
|
911
|
+<p>For a large cross-validation scheme, such as leave-2-out
|
|
|
912
|
+cross-validation, or when <em>results</em> contains many
|
|
|
913
|
+classifications, there are many feature set comparisons to make. Note
|
|
|
914
|
+that <em>rankingPlot</em> and <em>selectionPlot</em> have a
|
|
|
915
|
+<em>parallelParams</em> options which allows for the calculation of
|
|
|
916
|
+feature set overlaps to be done on multiple processors.</p>
|
|
|
917
|
+</div>
|
|
|
918
|
+<div id="generating-a-roc-plot" class="section level3">
|
|
|
919
|
+<h3>Generating a ROC Plot</h3>
|
|
|
920
|
+<p>Some classifiers can output scores or probabilities representing how
|
|
|
921
|
+likely a sample is to be from one of the classes, instead of, or as well
|
|
|
922
|
+as, class labels. This enables different score thresholds to be tried,
|
|
|
923
|
+to generate pairs of false positive and false negative rates. The naive
|
|
|
924
|
+Bayes classifier used previously by default has its <em>returnType</em>
|
|
|
925
|
+parameter set to <em>“both”</em>, so class predictions and scores are
|
|
|
926
|
+both stored in the classification result. So does diagonal LDA. In this
|
|
|
927
|
+case, a data frame with class predictions and scores for each class is
|
|
|
928
|
+returned by the classifier to the cross-validation framework. Setting
|
|
|
929
|
+<em>returnType</em> to <em>“score”</em> for a classifier which has such
|
|
|
930
|
+an option is also sufficient to generate a ROC plot. Many existing
|
|
|
931
|
+classifiers in other R packages also have an option that allows a score
|
|
|
932
|
+or probability to be calculated.</p>
|
|
|
933
|
+<p>By default, scores from different iterations of prediction are merged
|
|
|
934
|
+and one line is drawn per classification. Alternatively, setting
|
|
|
935
|
+<em>mode = “average”</em> will consider each iteration of prediction
|
|
|
936
|
+separately, average them and also calculate and draw confidence
|
|
|
937
|
+intervals. The default interval is a 95% interval and is customisable by
|
|
|
938
|
+setting <em>interval</em>.</p>
|
|
|
939
|
+<div class="sourceCode" id="cb44"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb44-1"><a href="#cb44-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ROCplot</span>(resultsList, <span class="at">fontSizes =</span> <span class="fu">c</span>(<span class="dv">24</span>, <span class="dv">12</span>, <span class="dv">12</span>, <span class="dv">12</span>, <span class="dv">12</span>))</span></code></pre></div>
|
|
|
940
|
+<p><img src="ClassifyR_files/figure-html/unnamed-chunk-17-1.png" width="576" /></p>
|
|
|
941
|
+<p>This ROC plot shows the classifiability of the asthma data set is
|
|
|
942
|
+high. Some examples of functions which output scores are
|
|
|
943
|
+<em>fisherDiscriminant</em>, <em>DLDApredictInterface</em>, and
|
|
|
944
|
+<em>SVMpredictInterface</em>.</p>
|
|
|
945
|
+</div>
|
|
|
946
|
+</div>
|
|
|
947
|
+<div id="other-use-cases" class="section level2">
|
|
|
948
|
+<h2>Other Use Cases</h2>
|
|
|
949
|
+<p>Apart from cross-validation of one data set, ClassifyR can be used in
|
|
|
950
|
+a couple of other ways.</p>
|
|
|
951
|
+<div id="using-an-independent-test-set" class="section level3">
|
|
|
952
|
+<h3>Using an Independent Test Set</h3>
|
|
|
953
|
+<p>Sometimes, cross-validation is unnecessary. This happens when studies
|
|
|
954
|
+have large sample sizes and are designed such that a large number of
|
|
|
955
|
+samples is prespecified to form a test set. The classifier is only
|
|
|
956
|
+trained on the training sample set, and makes predictions only on the
|
|
|
957
|
+test sample set. This can be achieved by using the function
|
|
|
958
|
+<em>runTest</em> directly. See its documentation for required
|
|
|
959
|
+inputs.</p>
|
|
|
960
|
+</div>
|
|
|
961
|
+<div id="cross-validating-selected-features-on-a-different-data-set"
|
|
|
962
|
+class="section level3">
|
|
|
963
|
+<h3>Cross-validating Selected Features on a Different Data Set</h3>
|
|
|
964
|
+<p>Once a cross-validated classification is complete, the usefulness of
|
|
|
965
|
+the features selected may be explored in another dataset.
|
|
|
966
|
+<em>previousSelection</em> is a function which takes an existing
|
|
|
967
|
+<em>ClassifyResult</em> object and returns the features selected at the
|
|
|
968
|
+equivalent iteration which is currently being processed. This is
|
|
|
969
|
+necessary, because the models trained on one data set are not directly
|
|
|
970
|
+transferrable to a new dataset; the classifier training (e.g. choosing
|
|
|
971
|
+thresholds, fitting model coefficients) is redone. Of course, the
|
|
|
972
|
+features in the new dataset should have the same naming system as the
|
|
|
973
|
+ones in the old dataset.</p>
|
|
|
974
|
+</div>
|
|
|
975
|
+<div id="parameter-tuning" class="section level3">
|
|
|
976
|
+<h3>Parameter Tuning</h3>
|
|
|
977
|
+<p>Some feature ranking methods or classifiers allow the choosing of
|
|
|
978
|
+tuning parameters, which controls some aspect of their model learning.
|
|
|
979
|
+An example of doing parameter tuning with a linear SVM is presented.
|
|
|
980
|
+This particular SVM has a single tuning parameter, the cost. Higher
|
|
|
981
|
+values of this parameter penalise misclassifications more. Moreover,
|
|
|
982
|
+feature selection happens by using a feature ranking function and then
|
|
|
983
|
+trying a range of top-ranked features to see which gives the best
|
|
|
984
|
+performance, the range being specified by a list element named
|
|
|
985
|
+<em>nFeatures</em> and the performance type (e.g. Balanced Accuracy)
|
|
|
986
|
+specified by a list element named <em>performanceType</em>. Therefore,
|
|
|
987
|
+some kind of parameter tuning always happens, even if the feature
|
|
|
988
|
+ranking or classifier function does not have any explicit tuning
|
|
|
989
|
+parameters.</p>
|
|
|
990
|
+<p>Tuning is achieved in ClassifyR by providing a variable called
|
|
|
991
|
+<em>tuneParams</em> to the SelectParams or TrainParams constructor.
|
|
|
992
|
+<em>tuneParams</em> is a named list, with the names being the names of
|
|
|
993
|
+the tuning variables, except for one which is named
|
|
|
994
|
+<em>“performanceType”</em> and specifies the performance metric to use
|
|
|
995
|
+for picking the parameter values. Any of the non-sample-specific
|
|
|
996
|
+performance metrics which <em>calcCVperformance</em> calculates can be
|
|
|
997
|
+optimised.</p>
|
|
|
998
|
+<div class="sourceCode" id="cb45"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb45-1"><a href="#cb45-1" aria-hidden="true" tabindex="-1"></a>tuneList <span class="ot"><-</span> <span class="fu">list</span>(<span class="at">cost =</span> <span class="fu">c</span>(<span class="fl">0.01</span>, <span class="fl">0.1</span>, <span class="dv">1</span>, <span class="dv">10</span>))</span>
|
|
|
999
|
+<span id="cb45-2"><a href="#cb45-2" aria-hidden="true" tabindex="-1"></a>SVMparams <span class="ot"><-</span> <span class="fu">ModellingParams</span>(<span class="at">trainParams =</span> <span class="fu">TrainParams</span>(<span class="st">"SVM"</span>, <span class="at">kernel =</span> <span class="st">"linear"</span>, <span class="at">tuneParams =</span> tuneList),</span>
|
|
|
1000
|
+<span id="cb45-3"><a href="#cb45-3" aria-hidden="true" tabindex="-1"></a> <span class="at">predictParams =</span> <span class="fu">PredictParams</span>(<span class="st">"SVM"</span>))</span>
|
|
|
1001
|
+<span id="cb45-4"><a href="#cb45-4" aria-hidden="true" tabindex="-1"></a>SVMresults <span class="ot"><-</span> <span class="fu">runTests</span>(measurements, classes, crossValParams, SVMparams)</span></code></pre></div>
|
|
|
1002
|
+<pre><code>## Processing sample set 10.</code></pre>
|
|
|
1003
|
+<pre><code>## Processing sample set 20.</code></pre>
|
|
|
1004
|
+<p>The index of chosen of the parameters, as well as all combinations of
|
|
|
1005
|
+parameters and their associated performance metric, are stored for every
|
|
|
1006
|
+validation, and can be accessed with the <em>tunedParameters</em>
|
|
|
1007
|
+function.</p>
|
|
|
1008
|
+<div class="sourceCode" id="cb48"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb48-1"><a href="#cb48-1" aria-hidden="true" tabindex="-1"></a><span class="fu">length</span>(<span class="fu">tunedParameters</span>(SVMresults))</span></code></pre></div>
|
|
|
1009
|
+<pre><code>## [1] 25</code></pre>
|
|
|
1010
|
+<div class="sourceCode" id="cb50"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb50-1"><a href="#cb50-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tunedParameters</span>(SVMresults)[<span class="dv">1</span><span class="sc">:</span><span class="dv">5</span>]</span></code></pre></div>
|
|
|
1011
|
+<pre><code>## [[1]]
|
|
|
1012
|
+## [[1]]$tuneCombinations
|
|
|
1013
|
+## topN cost Balanced Accuracy
|
|
|
1014
|
+## 1 10 0.01 0.8507719
|
|
|
1015
|
+## 2 20 0.01 0.8551553
|
|
|
1016
|
+## 3 30 0.01 0.8696398
|
|
|
1017
|
+## 4 40 0.01 0.9073756
|
|
|
1018
|
+## 5 50 0.01 0.8986087
|
|
|
1019
|
+## 6 60 0.01 0.8986087
|
|
|
1020
|
+## 7 70 0.01 0.8942253
|
|
|
1021
|
+## 8 80 0.01 0.9036592
|
|
|
1022
|
+## 9 90 0.01 0.9036592
|
|
|
1023
|
+## 10 100 0.01 0.8986087
|
|
|
1024
|
+## 11 10 0.10 0.8608729
|
|
|
1025
|
+## 12 20 0.10 0.8942253
|
|
|
1026
|
+## 13 30 0.10 0.8746903
|
|
|
1027
|
+## 14 40 0.10 0.9188107
|
|
|
1028
|
+## 15 50 0.10 0.9087097
|
|
|
1029
|
+## 16 60 0.10 0.9137602
|
|
|
1030
|
+## 17 70 0.10 0.9188107
|
|
|
1031
|
+## 18 80 0.10 0.9137602
|
|
|
1032
|
+## 19 90 0.10 0.9238613
|
|
|
1033
|
+## 20 100 0.10 0.9477797
|
|
|
1034
|
+## 21 10 1.00 0.8992758
|
|
|
1035
|
+## 22 20 1.00 0.8898418
|
|
|
1036
|
+## 23 30 1.00 0.9144273
|
|
|
1037
|
+## 24 40 1.00 0.9049933
|
|
|
1038
|
+## 25 50 1.00 0.9666476
|
|
|
1039
|
+## 26 60 1.00 0.9811321
|
|
|
1040
|
+## 27 70 1.00 0.9855155
|
|
|
1041
|
+## 28 80 1.00 1.0000000
|
|
|
1042
|
+## 29 90 1.00 1.0000000
|
|
|
1043
|
+## 30 100 1.00 1.0000000
|
|
|
1044
|
+## 31 10 10.00 0.9043263
|
|
|
1045
|
+## 32 20 10.00 0.8905089
|
|
|
1046
|
+## 33 30 10.00 0.9289118
|
|
|
1047
|
+## 34 40 10.00 0.9855155
|
|
|
1048
|
+## 35 50 10.00 1.0000000
|
|
|
1049
|
+## 36 60 10.00 1.0000000
|
|
|
1050
|
+## 37 70 10.00 1.0000000
|
|
|
1051
|
+## 38 80 10.00 1.0000000
|
|
|
1052
|
+## 39 90 10.00 1.0000000
|
|
|
1053
|
+## 40 100 10.00 1.0000000
|
|
|
1054
|
+##
|
|
|
1055
|
+## [[1]]$bestIndex
|
|
|
1056
|
+## [1] 28
|
|
|
1057
|
+##
|
|
|
1058
|
+##
|
|
|
1059
|
+## [[2]]
|
|
|
1060
|
+## [[2]]$tuneCombinations
|
|
|
1061
|
+## topN cost Balanced Accuracy
|
|
|
1062
|
+## 1 10 0.01 0.8066514
|
|
|
1063
|
+## 2 20 0.01 0.7783495
|
|
|
1064
|
+## 3 30 0.01 0.7877835
|
|
|
1065
|
+## 4 40 0.01 0.7783495
|
|
|
1066
|
+## 5 50 0.01 0.8117019
|
|
|
1067
|
+## 6 60 0.01 0.8117019
|
|
|
1068
|
+## 7 70 0.01 0.8117019
|
|
|
1069
|
+## 8 80 0.01 0.8261864
|
|
|
1070
|
+## 9 90 0.01 0.8261864
|
|
|
1071
|
+## 10 100 0.01 0.8261864
|
|
|
1072
|
+## 11 10 0.10 0.7928340
|
|
|
1073
|
+## 12 20 0.10 0.8029350
|
|
|
1074
|
+## 13 30 0.10 0.8406709
|
|
|
1075
|
+## 14 40 0.10 0.8406709
|
|
|
1076
|
+## 15 50 0.10 0.8457214
|
|
|
1077
|
+## 16 60 0.10 0.8551553
|
|
|
1078
|
+## 17 70 0.10 0.9181437
|
|
|
1079
|
+## 18 80 0.10 0.9326282
|
|
|
1080
|
+## 19 90 0.10 0.9275777
|
|
|
1081
|
+## 20 100 0.10 0.9326282
|
|
|
1082
|
+## 21 10 1.00 0.7746331
|
|
|
1083
|
+## 22 20 1.00 0.8602058
|
|
|
1084
|
+## 23 30 1.00 0.8652563
|
|
|
1085
|
+## 24 40 1.00 0.9023251
|
|
|
1086
|
+## 25 50 1.00 0.9413951
|
|
|
1087
|
+## 26 60 1.00 0.9514961
|
|
|
1088
|
+## 27 70 1.00 0.9521631
|
|
|
1089
|
+## 28 80 1.00 |
Dario Strbenac authored on 08/11/2022 06:00:07