AWS Amplify Removed Video Source

AWS Amplify removed video sources from HTML

A good amount the websites I’ve pushed to production in the past couple of years have all been hosted on-prem within my client’s data centers. However, not all of my clients have test environments. Up until recently I’ve been hosting my test sites with Server4You, a fantastic non AWS/Azure/GCP hosting company/provider. But with the rise of AWS and Azure I want to start venturing over that way to gain experience and play around with those providers some more.

I’m starting with a front end only React app that I’ve developed for a client which will ultimately live in their data center when the site goes live, but to test while we’re building the site I decided to give AWS Amplify a try. This service is awesome and hooks right into Gitlab for automated CI/CD with a couple button clicks. Awesome as it is, I did run into one problem: My video src references were stripped out during the CI/CD process.

The site has three HTML5 video references to mp4s that are hosted within the build folder at the root of the web project. When the AWS Amplify CI/CD process migrated my code over to the hosting web server all of my src references were blank – <video src="" />.. I tried referencing the videos in a couple different ways from the React app but nothing seemed to work.

To get the video links to work in AWS Amplify, I ended up creating a S3 storage account, uploading my videos to a S3 bucket , assigned the appropriate permissions then referencing the videos in code to pull from the S3 bucket which ended up fixing the problem.

I haven’t been able to find any documentation that explicitly says that video sources are removed but hosting the videos in S3 fixed the issue for me.

Filter DataTable By Column Value With Custom Dropdown Menu

In addition default search box in DataTables sometimes it’s nice to have the ability to filter by a specific DataTable column. This example shows how to use a custom drop-down menu to filter a DataTable by column value. I’m going to create a drop-down menu that displays the unique list of strings from a column called Category and when the user selects a category from the dropdown-menu, the datatable will be rendered with only records with the Category from the selected value.

Filter DataTable Example

View full working example.
Download full repo.

HTML

There are two main parts to the HTML, the category filter drop-down menu and the datatable. The values in the category filter will be the values that are to be filtered from the table when the user selects an item.

    <!-- Create the dropdown filter -->

   <div class="category-filter">
      <select id="categoryFilter" class="form-control">
        <option value="">Show All</option>
        <option value="Classical">Classical</option>
        <option value="Hip Hop">Hip Hop</option>
        <option value="Jazz">Jazz</option>
      </select>
    </div>

    <!-- Set up the datatable -->
    <table class="table" id="filterTable">
      <thead>
        <tr>
          <th scope="col">Artist</th>
          <th scope="col">Category</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td scope="col">Public Enemy</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Chet Baker</td>
          <td scope="col">Jazz</td>
        </tr>
        <tr>
          <td scope="col">Billie Holiday</td>
          <td scope="col">Jazz</td>
        </tr>
        <tr>
          <td scope="col">Vivaldi</td>
          <td scope="col">Classical</td>
        </tr>
        <tr>
          <td scope="col">Jurrasic 5</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Onyx</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Tchaikovsky</td>
          <td scope="col">Classical</td>
        </tr>
        <tr>
          <td scope="col">Oscar Peterson</td>
          <td scope="col">Jazz</td>
        </tr>
      </tbody>
    </table>

JavaScript

The JavaScript part relies on jQuery but can be modified to use vanilla javascript if you don’t have jQuery as part of the project. The code is commented below to give you an idea of what’s happening.

<script>
    $("document").ready(function () {

      $("#filterTable").dataTable({
        "searching": true
      });

      //Get a reference to the new datatable
      var table = $('#filterTable').DataTable();

      //Take the category filter drop down and append it to the datatables_filter div. 
      //You can use this same idea to move the filter anywhere withing the datatable that you want.
      $("#filterTable_filter.dataTables_filter").append($("#categoryFilter"));
      
      //Get the column index for the Category column to be used in the method below ($.fn.dataTable.ext.search.push)
      //This tells datatables what column to filter on when a user selects a value from the dropdown.
      //It's important that the text used here (Category) is the same for used in the header of the column to filter
      var categoryIndex = 0;
      $("#filterTable th").each(function (i) {
        if ($($(this)).html() == "Category") {
          categoryIndex = i; return false;
        }
      });

      //Use the built in datatables API to filter the existing rows by the Category column
      $.fn.dataTable.ext.search.push(
        function (settings, data, dataIndex) {
          var selectedItem = $('#categoryFilter').val()
          var category = data[categoryIndex];
          if (selectedItem === "" || category.includes(selectedItem)) {
            return true;
          }
          return false;
        }
      );

      //Set the change event for the Category Filter dropdown to redraw the datatable each time
      //a user selects a new filter.
      $("#categoryFilter").change(function (e) {
        table.draw();
      });

      table.draw();
    });
</script>

Full HTML/JS/CSS

Here is the full HTML with Javascript and a bit of CSS included.

<!doctype html>

<html lang="en">

<head>
  <meta charset="utf-8">

  <title>Add select drop-down filter to DataTable</title>
  <meta name="description" content="">
  <link rel="stylesheet" href="https://cdn.datatables.net/1.10.23/css/jquery.dataTables.min.css" />
  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" />
<style>
  select.form-control{
    display: inline;
    width: 200px;
    margin-left: 25px;
  }
</style>
</head>

<body>
  <div class="container mt-4">
    <!-- Create the drop down filter -->
    <div class="category-filter">
      <select id="categoryFilter" class="form-control">
        <option value="">Show All</option>
        <option value="Classical">Classical</option>
        <option value="Hip Hop">Hip Hop</option>
        <option value="Jazz">Jazz</option>
      </select>
    </div>

    <!-- Set up the datatable -->
    <table class="table" id="filterTable">
      <thead>
        <tr>
          <th scope="col">Artist</th>
          <th scope="col">Category</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td scope="col">Public Enemy</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Chet Baker</td>
          <td scope="col">Jazz</td>
        </tr>
        <tr>
          <td scope="col">Billie Holiday</td>
          <td scope="col">Jazz</td>
        </tr>
        <tr>
          <td scope="col">Vivaldi</td>
          <td scope="col">Classical</td>
        </tr>
        <tr>
          <td scope="col">Jurrasic 5</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Onyx</td>
          <td scope="col">Hip Hop</td>
        </tr>
        <tr>
          <td scope="col">Tchaikovsky</td>
          <td scope="col">Classical</td>
        </tr>
        <tr>
          <td scope="col">Oscar Peterson</td>
          <td scope="col">Jazz</td>
        </tr>
      </tbody>
    </table>

  </div>

  <script src="https://code.jquery.com/jquery-3.5.1.min.js"
    integrity="sha256-9/aliU8dGd2tb6OSsuzixeV4y/faTqgFtohetphbbj0=" crossorigin="anonymous"></script>

  <script src="https://cdn.datatables.net/1.10.23/js/jquery.dataTables.min.js"></script>

  <script>
    $("document").ready(function () {

      $("#filterTable").dataTable({
        "searching": true
      });

      //Get a reference to the new datatable
      var table = $('#filterTable').DataTable();

      //Take the category filter drop down and append it to the datatables_filter div. 
      //You can use this same idea to move the filter anywhere withing the datatable that you want.
      $("#filterTable_filter.dataTables_filter").append($("#categoryFilter"));
      
      //Get the column index for the Category column to be used in the method below ($.fn.dataTable.ext.search.push)
      //This tells datatables what column to filter on when a user selects a value from the dropdown.
      //It's important that the text used here (Category) is the same for used in the header of the column to filter
      var categoryIndex = 0;
      $("#filterTable th").each(function (i) {
        if ($($(this)).html() == "Category") {
          categoryIndex = i; return false;
        }
      });

      //Use the built in datatables API to filter the existing rows by the Category column
      $.fn.dataTable.ext.search.push(
        function (settings, data, dataIndex) {
          var selectedItem = $('#categoryFilter').val()
          var category = data[categoryIndex];
          if (selectedItem === "" || category.includes(selectedItem)) {
            return true;
          }
          return false;
        }
      );

      //Set the change event for the Category Filter dropdown to redraw the datatable each time
      //a user selects a new filter.
      $("#categoryFilter").change(function (e) {
        table.draw();
      });

      table.draw();
    });
  </script>
</body>

</html>

View full working example.
Download full repo.

Create multiple .env files in a React app

Configuring multiple environment files for a React app is handy if you have different variables for each environment (local, dev, test and production) – API endpoints is a good example. Like the name suggests, a .env file allows you to create environment specific variables that you specify to be used in specific builds for each of your environments. My projects usually have the four different environments:

  1. Local – My local machine where I do all my development.
  2. Development – This is the server environment where we can try new code outside our development machines. This is a server environment that is configured similar to test and production but is reserved to break things. Generally this environment is filled with dummy data and can become a mess pretty easily as we test out new features and debug code.
  3. Test or staging – A copy of the production environment where end users, testers and other folks from the team do their testing. This environment usually has the same data (minus any PII/PHI data) that production has.
  4. Production – This is the live site. You know what this is.

With these four environments I have five environment files in my React app structure. Each with their own variables that relate to the specific environment.

  • .env
    • This is a placeholder file that only shows the structure the other environment files should follow. For example:
      • REACT_APP_API_HOST=API_HOST
        REACT_APP_WEB_HOST=WEB_HOST
        REACT_APP_BUILD=BUILD
  • .env.development.local
    • This is the file that I use for my local development. I have this file added to .gitignore so it’s not checked into source control. Every developer should create this file locally and configure it with their local information.
  • .env.development
    • Development server
  • .env.staging
    • Staging server
  • .env.production
    • Production server

Then by utilizing the env-cmd package I’m able to run a command like npm run build:production to create my production build which uses the variables defined in the .env.production file. Same goes for local, development and testing/staging.

How to configure multiple .env files

Here’s how to set up a React application to utilize multiple .env files and variables. This example uses Create-React-App, so YMMV depending on what you bootstrapped your React app with.

1. Start off by creating a .env file at the root of your project for each of your environments.

Multiple .env files in React

2. Install the env-cmd package into your project npm install env-cmd

3. Open your package.json file and inside the scripts node add a line for each environment you will be building. You should already have lines for start, build, test and eject. Each line is specific to the build and .env file. So, build:development is the command you’ll run to create your development build, build: staging for testing and build:production for production.

"scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "build:development": "env-cmd -f .env.development react-scripts build",
    "build:staging": "env-cmd -f .env.staging react-scripts build",
    "build:production": "env-cmd -f .env.production react-scripts build"
  }

4. Add the environment specific variables to your environment files.

REACT_APP_API_HOST=https://localhost:5001
REACT_APP_WEB_HOST=http:localhost:3000
REACT_APP_BUILD=Development.Local

5. Run your build to create a build with your environment variables npm run build:production

Now you should have the ability to build your React apps with environment specific variables. Hopefully this makes your deployments easier and more straightforward.

There was an error running the selected code generator in .Net Core 5

I’m working on a new .Net Core 5 web app with user authentication and Identity Server 4 where I need to customize some of the Identity account pages (Login, Register, Forgot Password, etc). Out of the box these pages are built into .Net Core and Identity Server 4 so there’s nothing you need to do to use them. However, if you want to customize any of the account pages you’ll need to scaffold the source of those pages into your project.

In the latest version of Visual Studio 2019 – (Version 16.8.4 as of today) you are able to do this by right clicking the project and selected “Add –> New Scaffolded Item” for all .Net Core projects below version 5 without any issues. But if you are trying to scaffold Identity in .Net Core 5 from Visual Studio then you might run into this error:

“There was an error running the selected code generator: ‘Package restored failed. Rolling back package changes for ‘Your App’.”

A way around this is error I found was to use the dotnet CLI outside of Visual Studio. Follow these steps to scaffold the full Identity pages into your .Net Core 5 app.

  1. Close Visual Studio
  2. Open a command prompt and change directories to the project where you want to create the scaffolded items.
  3. Run the following command updating MyApp.Models.ApplicationDbContext to your own DbContext: dotnet aspnet-codegenerator identity -dc MyApp.Models.ApplicationDbContext

That should fix the “There was an error running the selected code generator” issue. You can also pass a set of parameters into the dotnet aspnet-codegenerator app if you only want to scaffold a certain number of files versus all Identity files.

Remove Passwords From Git History

Every once in a while you’ll check in a key or password into a git repository by mistake. Not to worry, there’s a great utility that can erase the values from history for you. You can delete the file or update the text from the current HEAD but the value still exists in your branch history so you need to go further to remove the value.

Luckily there’s the BFG Repo-Cleaner utility tool that goes through your git history and updates all references of the private value and replaces it with **Removed**.

Here’s how to use the BFG Repo-Cleaner to remove passwords from your git history. The BFG Repo-Cleaner is a Java file that can be saved to your local machine and run against your cloned git repo.

  1. Start by downloading the BFG Repo-Cleaner .jar file to your local machine. I saved mine to a folder called bfg on my D: drive – D:\code\bfg.
  2. Create a passwords.txt file that lists the passwords/keys you want to remove from your git repository and save to the same folder as the BFG Cleaner. The passwords.txt file is just a list of different passwords/keys that you want to remove from the repository.
    password1
    password2
    keyvalue123
  3. Clone your repository using the –mirror flag and make a full backup of the repo just in case something breaks.
    $ git clone --mirror https://github.com/clintmcmahon/delete-passwords.git
  4. Open command prompt and run the following command
    $ java -jar d:\code\bfg\bfg-1.13.0.jar --replace-text d:\code\bfg\passwords.txt d:\code\delete-passwords.git
  5. Move into the cloned git folder and run the following command
    $ cd delete-passwords.git
    $ git reflog expire --expire=now --all && git gc --prune=now --aggressive
  6. And finally push your changes back to the remote repository
    $ git push

That’s it. If you go into your repository now you will see that all references to your values from the passwords.txt file will now be replaced with **Removed**. This is just the tip of the utilities ability, there are more examples and other documentation on at the BFG Repo-Cleaner project Github page.

Sort Shopify Liquid Collection By A Nested Property

Suppose you want to sort or filter a Shopify collection by a sub/nested property. Well, this is an example of how you’d sort a Shopify collection by a nested property, specifically how to sort by a setting property of a logo list block.

Shopify tab example
Shopify filter tab example

I created a Shopify tab snippet to filter and sort the above list of logo blocks by a property within a single logo/block. The code below gets all the blocks in the logo list section, maps over the settings and returns the items where the logo_list property equals “brewery” then sorts the results by title. If you don’t need to return a specific sub-set of data and only want to sort by the property then simply remove | where: “logo_category”, “brewery” from the assign block.

{% assign breweries = section.blocks | map: 'settings' | where: "logo_category", "brewery" | sort: "title" %}

{% for block in breweries %}
    {%- assign logo = block.logo -%}
    {%- assign url = block.url -%}
        <div class="module-inline-item dynamic-logo-list-item" {{ block.shopify_attributes }}>
            {% if url != blank %}
              <a href="{{ url }}" target="_blank">
            {% endif %}

            {% if block.logo != blank %}
              {%
                include 'rimg',
                img: logo,
                size: '330x280',
                lazy: true
              %}
            {% else %}
              {{ 'logo' | placeholder_svg_tag: 'placeholder-svg' }}
            {% endif %}

            {% if url != blank %}
                </a>
            {% endif %}
          </div>
      {% endfor %}

And that’s all you need to do to filter a Shopify collection by a nested property. I will post the code for the full tab snippet in another post.

How To Create A Reusable Select List In React

This tutorial is on how to create a reusable dropdown list component in React. I broke the functionality down into two components — the actual select dropdown list component and the calling parent/calling component.

The parent component will be App.js and a component called DynamicSelect.js will handle the select list functionality. The entire source code project is on GitHub.

Let’s start with the DynamicSelect component. The DynamicSelect component is the component which will render an array of Seinfeld characters into select list and pass back the selected value via the props object to the parent component. When the onChange event is fired for the select list the event is passed into the handleChange function. This function will pass the selected value back to the parent (App.js) via the props object.

DynamicSelect.js Component

import React, {Component} from 'react';

class DynamicSelect extends Component{
    constructor(props){
        super(props)
    }

    //On the change event for the select box pass the selected value back to the parent
    handleChange = (event) =>
    {
        let selectedValue = event.target.value;
        this.props.onSelectChange(selectedValue);
    }

    render(){
        let arrayOfData = this.props.arrayOfData;
        let options = arrayOfData.map((data) =>
                <option 
                    key={data.id}
                    value={data.id}
                >
                    {data.name}
                </option>
            );

            return (
            <select name="customSearch" className="custom-search-select" onChange={this.handleChange}>
                <option>Select Item</option>
                {options}
           </select>
        )
    }
}

export default DynamicSelect;

App.js Component

import React, { Component } from 'react';
import logo from './logo.svg';
import './App.css';
import DynamicSelect from './DynamicSelect';

const arrayOfData = [
  {
    id: '1 - Jerry',
    name: 'Jerry'    
  },
  {
    id: '2 - Elaine',
    name: 'Elaine'    
  },
  {
    id: '3 - Kramer',
    name: 'Kramer'    
  },
  {
    id: '4 - George',
    name: 'George'    
  },
];

class App extends Component {
  constructor(props){
    super(props)
    this.state={
      selectedValue: 'Nothing selected'
    }
  }

  handleSelectChange = (selectedValue) =>{
    this.setState({
      selectedValue: selectedValue
    });
  }

  render() {
    return (
      <div className="App">
        <header className="App-header">
          <img src={logo} className="App-logo" alt="logo" />
          <h1 className="App-title">Dynamic Select Dropdown List</h1>
        </header>
        <p className="App-intro">
          <DynamicSelect arrayOfData={arrayOfData} onSelectChange={this.handleSelectChange} /> <br /><br />
          <div>
            Selected value: {this.state.selectedValue}
          </div>
        </p>
      </div>
    );
  }
}

export default App;

And that’s it. The repository for this code is available on GitHub in this class fashion. This functionatliy can be easily replicated via functional components with hooks as well.

Why I Write Unit Tests

In computer programming, unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use — wikipedia

Unit tests, who needs em? We all do. There are a lot of resources online that will tell you why unit testing is important or why writing unit tests before building implementations is a good idea. But I’m here to tell you how unit tests can save you a massive production headache.

A while back, a coworker pushed code to production that consumed and parsed large .csv files uploaded by users. The code looped through the file then parsed and mapped each record to the associated record in the model. The model contained a few bool fields that were populated by executing the C# Boolean.TryParse method against the text field to convert the string from the file to bool. If the TryParse method returned false then the value in the csv file must have been falsy, and vice versa for true. The record was applied to the model and saved to the database.

As it turned out, our code didn’t work as expected which caused a major problem. For bool fields users would upload either a 1 or 0. 1 being true and 0 being false. Because the source was a csv file all values were imported as strings. Therefore, before applying the values to our model, we must convert the string representation to the appropriate type within the model.

Here’s an example of the code, the Parse method below takes a string argument named parsable, does a Boolean.TryParse against parsable and returns the newly converted value.

	
	public static Main(){
	    var trueFalse = Parse("1");
	}
	
	public bool Parse(string parsable)
	{
		 bool flag;
		 bool result = Boolean.TryParse(parsable, out flag);
         return flag;
	}

From looking at the method and without knowing how C# Boolean.TryParse converts the string argument “1” to bool you’d think that this method returns true, not false. But in reality the above method returns false. Even though 1 generally equals true in most contexts, if passed in as a string the result is false, not true.

If we would have had proper unit tests set up we would have written a unit test that looks something like this:

[TestMethod]
public void Parse_BooleanTryParseString__ReturnsFalse()
{
    string stringToParse = "1";
    bool expected = true;
    bool actual = Parse(stringToParse);

    Assert.AreEqual(expected, actual);
}

The above unit test would have failed and we’d have found the problem, fixed it and been on our way. Unfortunately for us, we didn’t have a unit test for the above Parse method and QA didn’t catch the issue during their testing. The code eventually made its way to production, users start uploading files against the new code and yadda, yadda, yadda…we end up with a lot false values that should be true.

I've made a huge mistake

The good news is we were able to recover the lost data. But the recovery process took a few days and a lot of manual work which set us back on our timeline, not to mention created a garbage truck full of unwanted stress while we scrambled to recover the data and explain to our stakeholders that we’d made a huge mistake.

The takeaway my team and I had from this experience is how valuable a unit test can be. Unit tests are not just a key part of the testing process, but they’ve proven to be invaluable in making sure the code you write is actually doing what you think it’s doing.

Mapbox GL JS Access Denied In IE 11

Mapbox gives you the ability to load custom icons for different points in your maps. When I tried to load a custom icon to a Mapbox map I got the error “SCRIPT5: Access is denied.” in IE 11 – Firefox and Chrome loaded the icon fine.  My code was loading an imported image as an object inside a React component like this:

import centerImage from "./images/centerImage.png";

map.on('load', () => {
    map.loadImage(centerImage, (error, image) => {
        if (error) {
            return;
        }
        map.addImage('centerIcon', image);
    });
    ...
});

Updating the image URL parameter to a string URL fixed the problem. I didn’t dig too much into the issue to figure out why this was happening, but passing a valid image string URL was enough to get passed this issue.

map.on('load', () => {
    map.loadImage('/hospital.png', (error, image) => {
        if (error) {
            return;
        }
        map.addImage('centerIcon', image);
    });
    ...
});