full-text search using Lunr.js (#1726)

* maybe solved searching??

* Added comments for search

* attempt at a lunr search

* removed wrap terms func

* fixed lunr ref issue

* updated search for multilingual logic

* trying to fix enter bug

* trying list.js update function

* fixed earlier bug and got search working with filter logic

* fixed search bugs and got it working with filter logic

* fixed search bugs and got it working with filter logic

* added search enable button and more comments to code

* fixed most of the bugs in full text search

* added in support for multiple language stemmers

* create snippet templates for search terms

* updated search input to work with snippets

* trying to comment and clean up code

* FR snippets

* cleaned up code

* ordered lessons by search score

* removed abstract styling from style.css

* Update snippets.yml

translated to ES search snippets

* added search info and search indicator

* switched icon to question mark

* fixed search info for firefox and added loader

* cleaning up code and adding comments

* cleaning code and changing url to slug

* reverted back to url and added more comments

* removed all console.logs and added snippet

* ES translation of search documentation snippet

* FR translation search documentation snippet

* use md shortcut for URLs

* moved js files to vendor and only loaded file for language

* adding the vendor files again

* readding js vendor and updatinggitignore

Co-authored-by: Matthew Lincoln <[email protected]>
Co-authored-by: spapastamkou <[email protected]>
Co-authored-by: Jennifer Isasi <[email protected]>
Co-authored-by: Riva Quiroga <[email protected]>
Co-authored-by: Matthew Lincoln <[email protected]>

Author: 5 people

.gitignore

@@ -3,6 +3,6 @@ _site/*
 .DS_Store
 tmp/
 
-vendor/
+/vendor
 .bundle/
 .vscode/

_data/snippets.yml

@@ -352,6 +352,40 @@ filtering-results:
   es: Filtradas por
   fr: Filtrage par
   pt:
+loading-search:
+  en: LOADING SEARCH...
+  es: CARGANDO LA BÚSQUEDA...
+  fr: RECHERCHE EN COURS...
+  pt:
+search-lessons:
+  en: SEARCH LESSONS
+  es: BUSCAR LECCIONES 
+  fr: RECHERCHER LES LEÇONS
+  pt:
+start-searching:
+  en: START SEARCHING
+  es: INICIAR BÚSQUEDA
+  fr: DÉMARRER LA RECHERCHE
+  pt:
+type-search-terms:
+  en: TYPE SEARCH TERMS...
+  es: ESCRIBE LOS TÉRMINOS DE BÚSQUEDA...
+  fr: ENTRER UN OU PLUSIEURS TERMES DE RECHERCHE...
+  pt:
+search-info:
+  en: | 
+    To search for relevant terms, enter them into the search bar and either press the enter key or the search button. All searches are case insensitive. If multiple terms are entered, results will include lessons that have all terms, as well as lessons that contain *either term*. Results with more of the terms will be scored higher in the list.
+    
+    To only return results with multiple terms, add the `+` symbol before the term. For example, `+Twitter +Network` will return lessons that contain *both* "twitter" and "network". To only return results that *do not* contain a term use the `-` symbol. For example, `-Twitter +Network` will return lessons that contain "network" but not "twitter". For more detailed information, visit this guide on searching: <https://lunrjs.com/guides/searching.html>
+  es: | 
+    Para buscar términos relevantes, escríbelos en la barra de búsqueda y aprieta la tecla "enter" o haz clic en el botón "buscar lecciones". Las búsquedas no distinguen mayúsculas y minúsculas. Si se ingresan múltiples términos, los resultados incluirán tanto lecciones que los contengan todos, como lecciones que contengan *solo alguno*. Los resultados que incluyan más de un término aparecerán primero en la lista. 
+
+    Para obtener solo resultados con múltiples términos, agrega el símbolo `+` antes de cada uno. Por ejemplo, `+Twitter +Red` devolverá lecciones que contienen *tanto* "twitter" como "red". Para obtener solo resultados que *no* contengan cierto término, utiliza el símbolo `-`. Por ejemplo, `-Twitter +Red` devolverá lecciones que contienen "red" pero no "twitter". Para información más detallada, puedes revisar la guía sobre búsquedas (en inglés): <https://lunrjs.com/guides/searching.html>
+  fr: |
+    Pour obtenir des résultats en utilisant plusieurs termes de recherche, faites précéder chaque terme du symbole `+`. Par exemple, en insérant `+Twitter +réseau`, vous obtiendrez les tutoriels qui contiennent tous les deux termes. Pour *exclure* un terme des résultats obtenus, vous pouvez utiliser le symbole `-`. Par exemple, en insérant `-Twitter +réseau`, vous obtiendrez les tutoriels qui contiennent le terme "réseau", mais pas "Twitter". Pour en savoir plus sur les modalités de recherche, merci de consulter ce guide <https://lunrjs.com/guides/searching.html>
+
+  pt: |
+    
 
 # lesson headers
 editor:

_includes/lesson-index.html

@@ -1,32 +1,61 @@
 {% comment %}
- Core code for generating filterable lesson indexes on lessons.md and es/lecciones.md. This will repeatedly call lesson_describe.html in which individual lessons have their metadata formatted for display.
+Core code for generating filterable lesson indexes on lessons.md and es/lecciones.md. This will repeatedly call
+lesson_describe.html in which individual lessons have their metadata formatted for display.
 {% endcomment %}
-
+<!-- loading graph if search in URI -->
+<div id="pre-loader">
+  <svg width="200" height="200" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100" preserveAspectRatio="xMidYMid"
+    class="lds-ripple" style="background:0 0">
+    <circle cx="50" cy="50" r="4.719" fill="none" stroke="#1d3f72" stroke-width="2">
+      <animate attributeName="r" calcMode="spline" values="0;40" keyTimes="0;1" dur="3" keySplines="0 0.2 0.8 1" begin="-1.5s" repeatCount="indefinite" />
+      <animate attributeName="opacity" calcMode="spline" values="1;0" keyTimes="0;1" dur="3" keySplines="0.2 0 0.8 1" begin="-1.5s" repeatCount="indefinite" />
+    </circle>
+    <circle cx="50" cy="50" r="27.591" fill="none" stroke="#5699d2" stroke-width="2">
+      <animate attributeName="r" calcMode="spline" values="0;40" keyTimes="0;1" dur="3" keySplines="0 0.2 0.8 1" begin="0s" repeatCount="indefinite" />
+      <animate attributeName="opacity" calcMode="spline" values="1;0" keyTimes="0;1" dur="3" keySplines="0.2 0 0.8 1" begin="0s" repeatCount="indefinite" />
+    </circle>
+  </svg>
+</div>
 <ul class="filter activities">
   {% for activity in site.data.activities %}
   {% assign atype = activity.type %}
-  <li id="filter-{{ atype }}" class="filter">{{ activity[page.lang]}} ({{ alllessons | where:"activity",atype | size }})</li>
+  <li id="filter-{{ atype }}" class="filter">{{ activity[page.lang]}} ({{ alllessons | where:"activity",atype | size }})
+  </li>
   {% endfor %}
 </ul>
 
 <ul class="filter topics">
-{% for topic in site.data.topics %}
-{% assign type = topic.type %}
-{% assign count = alllessons | where:"topics",type | size }} %}
+  {% for topic in site.data.topics %}
+  {% assign type = topic.type %}
+  {% assign count = alllessons | where:"topics",type | size }} %}
   {% if count > 0 %}
   <li id="filter-{{ topic.type }}" class="filter">{{ topic.displayname[page.lang] }} ({{count}})</li>
   {% endif %}
-{% endfor %}
+  {% endfor %}
 </ul>
 
-<div id="filter-none">{{site.data.snippets.reset-button[page.lang]}} ({{  alllessons | size  }})</div>
+<div id="filter-none">{{site.data.snippets.reset-button[page.lang]}} ({{ alllessons | size  }})</div>
 
 <!--
 this div ('lesson-list', referenced in lessonfilter.js) needs to contain the sort button/elements AND the actual list for the sort buttons to work
 -->
+<div id="search-div" style="text-align: center; margin-bottom: 1rem; display: none;">
+  <input id="loading-search" class="search-input" type="text"
+    placeholder="{{ site.data.snippets.loading-search[page.lang] }}" style="background-color: #efefef" disabled>
+  <input id="search" class="search-input" type="text" style="display: none;" placeholder="{{ site.data.snippets.type-search-terms[page.lang] }}">
+  <button id="search-button" disabled>{{ site.data.snippets.search-lessons[page.lang] }}</button>
+  <i id="search-info-button" class="fas fa-question-circle"></i>
+  <div id="search-info">
+    {{ site.data.snippets.search-info[page.lang] | markdownify }}
+  </div>
+</div>
+<div id="enable-search-div" style="text-align: center; margin-bottom: 1rem;">
+  <button id="enable-search-button" style="width: 100%;">{{ site.data.snippets.start-searching[page.lang] }}</button>
+</div>
+
 <div id="lesson-list">
 
-<!--
+  <!--
 List.js uses button classes of asc and desc to control sorting functionality. It also toggles those classes off and on with click events.
 To not interfere with that, we use my-asc and my-desc to control the arrows, since our desired behavior is different from what list.js does.
 More concretely: to sort asc, the button class needs to contain asc, but we use the arrow ON THE BUTTON to indicate what WILL happen,
@@ -35,18 +64,22 @@
 -->
   <ul class="sort-by">
     <li id="sort-by-date" class="sort" data-sort="date">{{site.data.snippets.sort-by-date[page.lang]}}</li>
-    <li id="sort-by-difficulty" class="sort" data-sort="difficulty">{{site.data.snippets.sort-by-difficulty[page.lang]}}</li>
+    <li id="sort-by-difficulty" class="sort" data-sort="difficulty">{{site.data.snippets.sort-by-difficulty[page.lang]}}
+    </li>
   </ul>
 
   <input id="date-sort-text" type="hidden" label="{{site.data.snippets.date[page.lang]}}">
   <input id="difficulty-sort-text" type="hidden" label="{{site.data.snippets.difficulty[page.lang]}}">
 
-  <h2 class="results-title">{{ site.data.snippets.filtering-results[page.lang] }}: <span id="results-value">{{ site.data.snippets.all-lessons[page.lang] }} </span> <span id="current-sort" class="sort-desc">{{site.data.snippets.date[page.lang]}}</span></h2>
+
+  <h2 class="results-title">{{ site.data.snippets.filtering-results[page.lang] }}: <span
+      id="results-value">{{ site.data.snippets.all-lessons[page.lang] }} </span> <span id="current-sort"
+      class="sort-desc">{{site.data.snippets.date[page.lang]}}</span></h2>
 
   <ul class="list">
     {% for lesson in alllessons %}
     {% capture author_string %} {% include author.html %} {% endcapture %}
-      <li>{% include lesson_describe.html authors=author_string %}</li>
+    <li>{% include lesson_describe.html authors=author_string %}</li>
     {% endfor %}
   </ul>
 
@@ -56,10 +89,15 @@ <h2 class="results-title">{{ site.data.snippets.filtering-results[page.lang] }}:
 <script src='//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.3/jquery.min.js'></script>
 <script src='//cdnjs.cloudflare.com/ajax/libs/list.js/1.5.0/list.min.js'></script>
 <script src="{{ site.baseurl }}/js/URI.min.js"></script>
+<script src="https://unpkg.com/lunr/lunr.js"></script>
+<script src="{{ site.baseurl }}/js/vendor/lunr.stemmer.support.js"></script>
+{% if page.lang != "en" %}
+  <script src="{{ site.baseurl }}/js/vendor/lunr.{{page.lang}}.js"></script>
+{% endif %}
 <script src="{{ site.baseurl }}/js/lessonfilter.js"></script>
 
 <script>
-  $(function() {
+  $(function () {
     wireButtons();
   });
 </script>

_includes/lesson_describe.html

@@ -9,7 +9,6 @@
 {% endcomment %}
 
 <div class="lesson-description clearfix">
-
   {% if lesson.original %}
     {% assign canonical_slug = lesson.original %}
   {% else %}
@@ -23,7 +22,9 @@
   <h3 class="above-title">{{ include.authors }}</h3>
   <a href="{{ lesson.url }}"><h2 class="title">{{ lesson.title }}</h2></a>
 
-  <p class="abstract">{{ lesson.abstract | markdownify }}</p>
+  <p class="abstract">
+    {{ lesson.abstract | markdownify | remove: '<p>' | remove: '</p>'}}
+  </p>
 
   {% comment %}
  Activity and topic are hidden via CSS as a hack to let JS sorting by activity and topic work. Note that the date sort is based on translation date first, falling back to publication date.
@@ -32,5 +33,13 @@ <h3 class="above-title">{{ include.authors }}</h3>
   <span class="topics">{{ lesson.topics | join: ' '}}</span>
   <span class="date">{% if lesson.translation_date %}{{ lesson.translation_date }}{% else %}{{ lesson.date }}{% endif %}</span>
   <span class="difficulty">{{ lesson.difficulty }}</span>
+  <span id="{{lesson.url}}-content" class="content"
+    style="display: none;">{{ lesson.content | markdownify | strip_html }}</span>
+  <span id="{{lesson.url}}-score" class="score"
+    style="display: none;">0</span>
+  <p id="{{lesson.url}}-search_results" class="search_results"
+    style="display: none;"></p>
+  
+
 
 </div>

css/style.css

@@ -316,6 +316,74 @@ Lessons Index
     background-color:#ededed;
   }
 
+  /*****************
+    SEARCH
+  *****************/
+
+  .search-input {
+    width:55%;
+    clear:both;
+    margin-bottom:1rem;
+    background-color:#fefefe;
+    color:#666;
+    border:1px solid #999;
+    font: .9rem/1.1rem 'Open Sans',
+    sans-serif;
+    padding: .4rem .6rem;
+    border-radius: 3px;
+    display:inline-block;
+    text-transform:uppercase;
+    text-decoration: none;
+  }
+
+  #search-button,
+  #enable-search-button {
+    background-color: #efefef;
+    color: rgb(153, 143, 143);
+    width: 35%;
+    font: .9rem/1.1rem 'Open Sans',
+    sans-serif;
+    padding: .4rem .6rem;
+    border: none;
+    border-radius: 3px;
+    display: inline-block;
+    text-transform: uppercase;
+    text-decoration: none;
+  }
+
+  @media only screen and (max-width: 767px) {
+    /* phones */
+    #search-button,
+    #enable-search-button {
+      width: 80%;
+    }
+  }
+
+
+  #search-info-button {
+    padding: 0.5rem;
+    color: rgb(153, 143, 143);
+  }
+
+  #search-info {
+    display: none;
+    height:0px;
+    background:#efefef;
+    overflow:hidden;
+    transition:0.5s;
+    -webkit-transition:0.5s;
+    width: 100%;
+    text-align: left;
+    box-sizing: border-box;
+  }
+
+  #search-info.visible {
+    display: block;
+    height: fit-content;
+    height: -moz-max-content;
+    padding: 10px;
+    margin-top: 10px;
+  }
 
   /*****************
     SORT BUTTONS
@@ -426,19 +494,27 @@ Lessons Index
     clear:none;
   }
 
-  p.abstract {
-    font: .8rem/1.2rem 'Open Sans', sans-serif;
-    margin:0;
-  }
-
   .list .date,
   .lesson-description .activity,
   .lesson-description .topics,
   .lesson-description .difficulty {
     display: none;
   }
 
-
+  #pre-loader {
+    visibility: hidden;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    height: 100vh;
+    width: 100%;
+    position: fixed;
+    top: 0;
+    left: 0;
+    z-index: 9999;
+    transition: opacity 0.3s linear;
+    background: rgba(211, 211, 211, 0.8);
+  }
 /* =============================================================================
 Top Navigation Bar
 ========================================================================== */