{"id":440,"date":"2021-07-31T16:29:04","date_gmt":"2021-07-31T14:29:04","guid":{"rendered":"https:\/\/mirblog.me\/?p=440"},"modified":"2021-07-31T16:32:44","modified_gmt":"2021-07-31T14:32:44","slug":"development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python","status":"publish","type":"post","link":"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/","title":{"rendered":"Development and Release of Type4Py: Machine Learning-based Type Auto-completion for Python"},"content":{"rendered":"<p style=\"text-align: justify;\">Over the past decade, machine learning (ML) has been applied successfully to a variety of tasks such as computer vision and natural language processing. Motivated by this, in recent years, researchers have employed ML techniques to solve code-related problems, including but not limited to, code completion, code generation, program repair, and type inference.<\/p>\n<p style=\"text-align: justify;\">Dynamic programming languages like Python and TypeScript allows developers to optionally define type annotations and benefit from the advantages of static typing such as better code completion, early bug detection, and etc. However, retrofitting types is a cumbersome and error-prone process. To address this, we propose <em>Type4Py<\/em>, an ML-based type auto-completion for Python. It assists developers to gradually add type annotations to their codebases. In the following, I describe Type4Py&#8217;s pipeline, model, deployments, and the development of its VSCode extension and more.<\/p>\n<p><!--more--><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-69d427dceef31\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-69d427dceef31\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Type4Py\" >Type4Py<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Overview\" >Overview<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Dataset\" >Dataset<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Feature_Extraction\" >Feature Extraction<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Model_Architecture_Training\" >Model Architecture &amp; Training<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Implementation\" >Implementation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Deployment\" >Deployment<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#VSCode_Extension\" >VSCode Extension<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Releasing\" >Releasing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Roadmap\" >Roadmap<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#Acknowledgments\" >Acknowledgments<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/#References\" >References<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Type4Py\"><\/span>Type4Py<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3><span class=\"ez-toc-section\" id=\"Overview\"><\/span>Overview<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">Before going into the detail of the Type4Py&#8217;s model and its implementation, it would be helpful to see the overview of Type4Py. In general, there is a VSCode extension at the client-side (developers) and the Type4Py model and its pipeline are deployed on our servers. Simply, the extension sends a Python file to the server and a JSON response containing type predictions for the given file is returned. In the following subsection, I describe each component.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-thumbnail\" src=\"https:\/\/github.com\/saltudelft\/type4py-vscode-ext\/raw\/master\/images\/design.png\" alt=\"Type4Py's overview\" width=\"1908\" height=\"1088\" \/><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Dataset\"><\/span>Dataset<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">For an ML model to learn and generalize, there is a need for a large and high-quality dataset. Therefore, we created the <a href=\"https:\/\/zenodo.org\/record\/4719447\">ManyTypes4Py<\/a> dataset which contains 5.2K Python projects and 4.2M type annotations. To avoid data leakage from the training set to the test set, we used our <a href=\"https:\/\/github.com\/saltudelft\/CD4Py\">CD4Py<\/a> tool to perform code de-duplication in the dataset. To know more about how we created the dataset, check out <a href=\"https:\/\/arxiv.org\/abs\/2104.04706\">its paper<\/a>.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Feature_Extraction\"><\/span>Feature Extraction<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">As with most ML tasks, we need to find a set of relevant features in order to predict type annotations. Here, we consider features as type hints. Specifically, we extract three kinds of type hints, namely, identifiers, code context, and visible type hints (VTHs). To intuitively explain the type hints, consider below code snippet:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nimport numpy as np\r\ndef find_list_intersection(list_a, list_b):\r\n    intersec_list = &#x5B;]\r\n        for i in list_a:\r\n            if i in list_b:\r\n                intersec_list.append(i)\r\nreturn np.array(intersec_list)\r\n<\/pre>\n<p style=\"text-align: justify;\">We use the function&#8217;s name, its parameters&#8217; name, and variables&#8217; name as type hints to predict types. Considering code context, we consider the usage of parameters and variables in statements that they are used. VTHs are a deep recursive analysis of import statements in a file and its transitive dependencies in our dataset. Later on, we build a VTH vocabulary, which is used to give a hint to the model about the expected type.<\/p>\n<p>Given the above code snippet, we create the following sequences for identifiers and code context to predict types of variables, parameters, and the function&#8217;s return:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\n# For identifiers\r\nvar_id_seq = &#x5B;'intersec_list']\r\n# Code context\r\nvar_cc_deq = &#x5B;'intersec_list', 'append', 'i']\r\nparam_id_seq = &#x5B;'list_a', 'find_list_intersection',\r\n                'list_b'] \r\nparam_cc_seq = &#x5B;'i', 'list_a']\r\n# Guess the identifer and code context sequences\r\n# for list_b!\r\nret_id_seq = &#x5B;'find_list_intersection', 'list_a',\r\n              'list_b']\r\nret_cc_seq = &#x5B;'np' 'array', 'intersec_list']\r\n<\/pre>\n<p>To learn from the extracted sequences, first, we apply common NL pre-processing techniques tokenization (by <a href=\"https:\/\/en.wikipedia.org\/wiki\/Snake_case\">snake case<\/a>), stop word removal and lemmatization. Then, we employ the famous Word2Vec model to generate word embeddings of the extracted sequences in order to train the Type4Py model, which is described next.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Model_Architecture_Training\"><\/span>Model Architecture &amp; Training<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Before dive deep into the model&#8217;s details, an overview of the Type4Py model is shown below:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-463 size-full\" src=\"https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06.png\" alt=\"\" width=\"3158\" height=\"914\" srcset=\"https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06.png 3158w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-300x87.png 300w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-1024x296.png 1024w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-768x222.png 768w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-1536x445.png 1536w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-2048x593.png 2048w, https:\/\/mirblog.net\/wp-content\/uploads\/2021\/07\/Screenshot-2021-07-28-at-18.28.06-1200x347.png 1200w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p style=\"text-align: justify;\">The Type4Py model is a hierarchical neural network, which consists of two RNNs with LSTM units, one for identifiers and another for code context. The basic idea is that two RNN captures different aspects of input sequences from both identifiers and code context. Next, the output of two RNNs is concatenated into a single vector, which is passed through a fully-connected linear layer. The final linear layer maps the learned type annotation into a high-dimensional feature space, called Type Clusters. In order to create Type Clusters, we need to formulate the type prediction task as a similarity learning problem, rather than a classification problem. Intuitively, the model learns to map similar types (say float) into its own type cluster and it should be as far as possible from the type cluster of other dissimilar types.<\/p>\n<p style=\"text-align: justify;\">Considering the described similarity learning problem, the Type4Py model is trained using the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Triplet_loss\">Triplet loss<\/a> function, which is recently used in computer vision for face recognition. Simply, the model receives three training examples, a to-be-learned type annotation (called anchor), a training example same as the anchor, a training example with a different type annotation from the first and second examples. For example, Let&#8217;s assume that the first training sample in the training set is the type annotation float plus its vector representations. Then, we randomly find a training sample with the float type annotation in the training and a training sample with a different type annotation. In this example, the triplet the model learns from would be (float, float, str).<\/p>\n<p style=\"text-align: justify;\">After the model is trained and type clusters are obtained, to infer a type for a given query, we perform \\(k\\)-nearest neighbor (KNN) search in the type clusters to suggest a list of similar types for the given query. The value of \\(k\\) is \\(10\\) in both research and production environments.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Implementation\"><\/span>Implementation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">Our server-side components are all written in Python. To extract type hints, we first extract Abstract Syntax Trees (ASTs) and perform light-weight static analysis using our <a href=\"https:\/\/github.com\/saltudelft\/libsa4py\">LibSA4Py<\/a> package. NLP tasks are applied using NLTK. To train the Word2Vec model, we use the <a href=\"https:\/\/radimrehurek.com\/gensim\/\">gensim<\/a> package.<\/p>\n<p style=\"text-align: justify;\">For the Type4Py model, we use bidirectional LSTMs in <a href=\"https:\/\/pytorch.org\/\">PyTorch<\/a> to implement the two RNNs. To avoid overfitting, we apply the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Dilution_(neural_networks)\">Dropout regularization<\/a> to the input sequences. To minimize the value of the Triplet loss function, we employ the Adam optimizer. Also, to speed up the training process, we use the data parallelism feature of PyTorch, which distributes training batches among GPUs. For fast KNN search, we use <a href=\"https:\/\/github.com\/spotify\/annoy\">Annoy<\/a>.<\/p>\n<p style=\"text-align: justify;\">To process type prediction requests from users, we implemented a tiny web application in Flask. It features type prediction using the Type4Py model, storing telemetry data from the VSCode extension (based on the user&#8217;s consent), and API rate limit. Lastly, we employ <a href=\"https:\/\/mypyc.readthedocs.io\/en\/latest\/\">mypy<\/a> to type-check the model&#8217;s predictions. However, as of this writing, the type-checking component is disabled for performance reasons. The Type4Py&#8217;s model and its server code are publicly available on GitHub <a href=\"https:\/\/github.com\/saltudelft\/type4py\">here<\/a>.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Deployment\"><\/span>Deployment<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">To deploy the Type4Py model for the production environment, we convert the pre-trained PyTorch model to an <a href=\"https:\/\/www.onnxruntime.ai\/\">ONNX<\/a> model which allows us to query the model on both GPUs and CPUs with very fast inference speed and lower VRAM consumption. Thanks to Annoy, Type Clusters are memory-mapped into RAM from disk, which consumes less memory.<\/p>\n<p style=\"text-align: justify;\">To handle concurrent type prediction requests from users, we employ <a href=\"https:\/\/gunicorn.org\/\">Gunicorn<\/a>&#8216;s HTTP server with Nginx as a proxy. This allows us to have quite a number of asynchronous workers that have an instance of Type4Py&#8217;s ONNX model plus Type Clusters each. We use <a href=\"http:\/\/supervisord.org\/\">supervisor<\/a> to manage and monitor the deployment.<\/p>\n<p style=\"text-align: justify;\">We deployed the whole Type4Py&#8217;s server-side application on a Linux machine with Ubuntu distribution. The server has an Intel Xeon CPU with 28 threads, 128 GB of RAM, and two Nvidia GTX 1080 TIs. Lastly, we test our server&#8217;s production code with integration tests using GitHub Workflow.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"VSCode_Extension\"><\/span>VSCode Extension<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The Type4Py&#8217;s VSCode extension is small and simple. Here is an overview of the extension workflow in VSCode:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-thumbnail\" src=\"https:\/\/raw.githubusercontent.com\/saltudelft\/type4py-vscode-ext\/master\/images\/extension-quick-start.gif\" alt=\"Type4Py's VSCode Extension\" width=\"960\" height=\"540\" \/><\/p>\n<p style=\"text-align: justify;\">In general, to get type predictions, the extension sends an opened Python source file to the server and receives a JSON response. To implement the extension, we use the <a href=\"https:\/\/code.visualstudio.com\/api\/references\/vscode-api\">VSCode API<\/a> in TypeScript. More specifically, we employ IntelliSense&#8217;s completion functionality and some pattern matching to locate type slots in the code for suggesting likely type annotations. Type slots are functions parameters, return types, and variables, which are located based on the line and column numbers. Currently, type prediction can be triggered via Command Pallete or by enabling the AutoInfer setting, which predicts types whenever a Python source file is opened or selected. Finally, the extension gathers telemetry data from users based <strong>on their consent<\/strong>. For instance, telemetry data contains which type predictions are accepted or rejected at a file&#8217;s type slots for research purposes and improving the Type4Py model. The Type4Py&#8217;s VSCode extension can be installed from the VS Marketplace <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=saltud.type4py\">here<\/a>.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Releasing\"><\/span>Releasing<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">We have two different environments for development and production as it is common in software development. In the development env., we test, debug, and profile Type4Py&#8217;s server-side components before releasing new features\/fixes into the production code. Therefore, users will not be affected by new changes and features in the production env. Also, whenever we train a new Type4Py neural model, we test it against its evaluation metrics (see <a href=\"https:\/\/arxiv.org\/abs\/2101.04470\">its paper<\/a>) and run integration tests to ensure that it produces expected predctions for given Python source files. Finally, the VSCode extension uses the development env. when testing new featues\/fixes before releasing a new extension version.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Roadmap\"><\/span>Roadmap<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"text-align: justify;\">So far, I have described the current state of Type4Py. For future work, here is our roadmap:<\/p>\n<ul>\n<li style=\"text-align: justify;\">Enabling the type-checking process for the Type4Py&#8217;s predictions using <a href=\"https:\/\/github.com\/python\/mypy\">mypy<\/a>, preferably at the client-side.<\/li>\n<li style=\"text-align: justify;\">Releasing a local version of the Type4Py model and its pipeline that can be queried on users&#8217; machines.<\/li>\n<li style=\"text-align: justify;\">Fine-tuning the (pre-trained) Type4Py model on users&#8217; projects to learn project-specific types.<\/li>\n<li>Releasing a plugin for JetBrains PyCharm and a GitHub action or bot for adding type annotations to Python projects<\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"Acknowledgments\"><\/span>Acknowledgments<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\">The Type4Py model, its pipeline, and VSCode extension are all designed and developed at the <a href=\"https:\/\/github.com\/saltudelft\">Software Analytics Lab<\/a> of <a href=\"https:\/\/se.ewi.tudelft.nl\/\">SERG<\/a>, Delft University of Technology. Our team consists of 4 researchers and developers, <a href=\"https:\/\/mirblog.net\/\">Amir M. Mir<\/a>, <a href=\"https:\/\/elatoskinas.github.io\/\">Evaldas Lato\u0161kinas<\/a>, <a href=\"https:\/\/proks.ch\/\">Sebastian Proksch<\/a>, <a href=\"https:\/\/www.gousios.gr\/\">Georgios Gousios<\/a>. The Type4Py project has started in spring 2020 and it&#8217;s currently under active development and research.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-thumbnail\" src=\"https:\/\/github.com\/saltudelft\/type4py-vscode-ext\/blob\/master\/images\/tudlogo.png?raw=true\" width=\"250\" height=\"150\" \/><\/p>\n<h2><span class=\"ez-toc-section\" id=\"References\"><\/span>References<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Here are a few resources for further reading and research:<\/p>\n<ol>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2101.04470v2\">Type4Py&#8217;s research paper<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2104.04706\">ManyTypes4Py dataset&#8217;s paper<\/a> (accepted at MSR&#8217;21)<\/li>\n<li><a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=saltud.type4py\">Type4Py&#8217;s VSCode Extension on the VS Marketplace<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/saltudelft\/libsa4py\">LibSA4Py: Light-weight static analysis for extracting type hints and features<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/saltudelft\/CD4Py\">CD4Py: Code De-Duplication for Python<\/a><\/li>\n<li><a href=\"https:\/\/code.visualstudio.com\/api\/get-started\/your-first-extension\">Development of VSCode Extensions<\/a><\/li>\n<li><a href=\"https:\/\/pytorch.org\/tutorials\/advanced\/super_resolution_with_onnxruntime.html\">Converting PyTorch models to ONNX format<\/a><\/li>\n<li><a href=\"https:\/\/youtu.be\/FSL3hvpd4cI\">Type4Py vs. Tabnine: An ML-based type auto-completion comparison<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/saltudelft\/ml4se\">Machine Learning for Software Engineering research<\/a><\/li>\n<\/ol>\n<p style=\"text-align: justify;\">Thanks for reading this blog post. I hope that it was useful and informative for readers. We would be glad to answer your questions and see your comments. Also, please share this blog post with your colleagues and friends if you think they might be interested.<\/p>\n\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Over the past decade, machine learning (ML) has been applied successfully to a variety of tasks such as computer vision and natural language processing. Motivated by this, in recent years, researchers have employed ML techniques to solve code-related problems, including but not limited to, code completion, code generation, program repair, and type inference. Dynamic programming &hellip; <a href=\"https:\/\/mirblog.net\/index.php\/2021\/07\/31\/development-and-release-of-type4py-machine-learning-based-type-auto-completion-for-python\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Development and Release of Type4Py: Machine Learning-based Type Auto-completion for Python&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,32],"tags":[83,67,70,63,80,78,75,74,71,28,69,72,76,24,73,82,79,77,81,66,64,65,62,84,68],"class_list":["post-440","post","type-post","status-publish","format-standard","hentry","category-machine-learning","category-research","tag-artificial-intelligence","tag-auto-completion","tag-dataset","tag-deep-learning","tag-developer-productivity","tag-development","tag-extension","tag-knn","tag-libsa4py","tag-machine-learning","tag-manytypes4py","tag-onnx","tag-plugin","tag-python","tag-pytorch","tag-serg","tag-software-engineering","tag-static-types","tag-tudelft","tag-type-annotations","tag-type-inference","tag-type-prediction","tag-type4py","tag-visual-studio","tag-vs-code"],"_links":{"self":[{"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/posts\/440","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/comments?post=440"}],"version-history":[{"count":40,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/posts\/440\/revisions"}],"predecessor-version":[{"id":484,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/posts\/440\/revisions\/484"}],"wp:attachment":[{"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/media?parent=440"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/categories?post=440"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mirblog.net\/index.php\/wp-json\/wp\/v2\/tags?post=440"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}