Planet CDOT (Telescope)

Sunday, February 5, 2023


Taimoor Dawami

OSD700 - Eventful week contributing to Starchart

Last week was all about getting started; this week, we were tasked with building on top of the Blues Stack template we set up last week.

This week was quite eventful, and we broke things, but we also managed to fix things as a team. There was a lot of collaboration and active participation, which needs to be improved in some of the other courses I have previously taken.

There were moments when I felt stuck or made a mistake that I thought was irreversible, but luckily the team I am working with was there to steer me in the right direction.

My first task with porting over middleware used by Satellite and adding it to Remix's server. Remix's Blues Stack, by default, uses Morgan, but we wanted to use Pino instead. However, Pino seems to cause a problem, so David has to switch to using Winston.

My second task was to configure Nodemailer to send email notifications in the future. We have set up the function responsible for sending emails, but there's no way of knowing if it will work. We may have to make some adjustments to the current Nodemailer configuration. Next week, I aim to implement functionality to ensure we can successfully send test email notifications.

Finally, setting up Dependabot was fun, but some of us got bothered by it, while others remained unaffected. One of the significant issues with the Dependabot setup was that it would raise PRs for major version updates. We do not want Dependabot to major version updates, as, according to David, it's risky and should be handled manually. Gladly, I resolved that issue by restricting Dependabot to minor and patch-level version updates.

This week I want to ensure we can send the test notifications successfully. Also, I want to enhance Dependabot's current configuration. I am open to helping in other areas of development if needed.

by Taimoor Dawami at Sun Feb 05 2023 22:35:29 GMT+0000 (Coordinated Universal Time)


Stefan Frunza

Remixing the Blues

Starting to contribute

This week on the starchart project we set out what we were going to commit to completing for the 0.1 Milestone. This is now concrete coding work we are talking about, along with continuously reading documentation and now also peer-reviewing PRs and helping one another get setup.

In the very beginning of the week I had some issues getting the starchart project running on my machine and the team got together on a call after our first scheduled meeting and helped me out. I learned that I had to build before running and that my mysql-data folder had some issues so it had to be deleted and the generated again.

I had problems again later, eventually after bouncing ideas back and forth with TD on how to solve it it turns out deleting my node modules and re installing worked (just npm installing without deleting did not). Still not sure why but in any case, I was able to work with Star Chart and I got to doing the small part that I had set out to.

Initial Auth

The issue I had commited to was to create a login button that would authenticate as the single user that is seeded into the dev database, I then also created a log out button that would destroy the session.

Initially I had thought that I would have to write the session.server.ts file and make my own createCookieSession methods to work with, docs for which can be found here. This did not turn out to be the case because apparently the blues stack already has a really great session.server.ts file with some really nice methods such as some require methods that will automatically throw redirects to login routes if conditions aren't met and things of that nature which I made use of.

There is also a longer example that takes you through the creation of a joke application that has helped me work with remix here.

The logic that I use comes from these docs.

What I did was I added 2 files login.tsx and logout.tsx under the routes folder. The login.tsx has an action that occurs upon the press of the login button that will use the prewritten createUserSession() method with the seeded users username and redirect to the Index page.

I then edited the index page to include a loader that would require the authenticated user, there was yet again a really helpful pre-written function to use from session.server.ts requireUsername() that checks the request from Loader Arguments for the username and if it does not find it it redirects to login. If it does find it it returns the user's username to the Index component to be able to make use of it when it renders the HTML.

I then added a button to the Index HTML that would use the action of the new logout route when pressed.

The logout route has the action I just mentioned which uses yet another pre-written function that is called logout() which destroys the session and it also has a loader just in case someone accidentally navigates to it. The loader simply redirects the client to the base Index route.

PRs and Reviews

Here are the 2 PRs that I made this week and the comments and changes associated with them.

Login and Logout
Changing User Model after Prisma Schema change

The second one was a small one to change the user model and seed to match the changes to the Prisma Schema whereby name was split into first and last names.

I also did a few approvals on other students PRs. I really did not know how to improve them, they seemed good to me and that's what I wrote. I hate to not be able to contribute more but I also hate to see PRs that look really well done sitting there blocked with only one review.

Thanks for reading. Onto reading more about SAML.

by Stefan Frunza at Sun Feb 05 2023 20:38:30 GMT+0000 (Coordinated Universal Time)


David Humphrey

Starchart 0.1!

Later today we're going to be shipping our first release of Starchart, our custom-domain and certificate web app for the Seneca community.  I was telling my boss about it this week, and she was interested to hear that our students are building all this.  I wanted to talk about what it includes and how we built it.

Eventually, Starchart is going to allow Seneca users to create and manage their own custom domain names and SSL certificates via a custom web app.  We're building it using Remix.run, Chakra-UI, Amazon Route53, Let's Encrypt, Prisma and MySQL, Docker, GitHub Actions, Vitest and Playwright.

The 0.1 Release is creating the foundation on which we'll build everything.  There was a lot to set up. Here are some of the major themes and highlights of what it includes:

  • Configured and customized the Remix.run Blue Stack as our starting point
  • Setup MySQL in Docker and got our database/ORM schema created
  • Wrote a lot of docs, including our contributing guide, updated our README, and added more technical docs to the wiki
  • Setup our CI pipeline, with a lot of cool optimizations (parallel build jobs, intelligent caching, service containers, etc). We also tracked down and fixed a bunch of CI related bugs this week
  • Setup and tweaked our dependency management with Dependabot, which has already landed 3 fixes, and has another 5 in review
  • Setup Chakra UI and created a default theme
  • Started to design our web app in Figma using the Chakra UI Figma components.  Here's an in-progress example:
User Domains Dashboard
  • Created a module for working with Let's Encrypt, and landed the first pieces of functionality for creating a certificate order and extracting challenges
  • Created a module for working with Route53, and implemented all client API calls with tests!
  • Created a module for sending email notifications via Seneca's Office365 using OAuth2
  • Configured a bunch of developer goodies to make our work easier (e.g., prettier githook, VSCode integrations, proper logging in dev vs. prod, eslint rules to make reviews go faster, etc)
  • Implemented initial cookie-session based login/logout in the back- and front-end
  • Improved our server's security
  • Wrote our first E2E tests using Playwright and got them working in CI.  They will even upload detailed HTML reports and videos when tests fail!
  • Added MySQL as well as mock Route53 and Let's Encrypt containers for development and testing
  • Began work to create our dockerized deployment webhook
  • Worked with Seneca ITS to create our deployment strategy for staging and production, registered domains, and created a bunch of accounts for AWS, MySQL, Azure Active Directory, etc.  Mehrdad has been really helpful to work with.

Along the way we reviewed and closed over 50 pull requests and landed code written by 12 contributors. We did a ton of reviews, broke and fixed the main branch several times, and got to help each other find our way around a new and rapidly changing code base. We also worked well as a team:

Alex on Slack

I'm looking forward to seeing what we'll do by 0.2.

by David Humphrey at Sun Feb 05 2023 16:05:09 GMT+0000 (Coordinated Universal Time)

Saturday, February 4, 2023


Denes Adam Dolhay

Week 4, creating an order

This week I have worked on creating an order with Let's encrypt in manual mode.

Main work

The most appropriate format for the code seemed to be an ES6 class.
The class works as follows:

  • The constructor creates a new empty instance
  • the initialize async instance function instantiates a let's encrypt client (node-acme-client instance) and stores it in a private instance variable. It also verifies / creates a let's encrypt account based on the PEM that it got in a docker secret
  • the createOrder async instance function is then called, which creates an order with the provider, detailing all the identifiers (domains) that the certificate should sign. It then calls internally another function, that loops through all of the "authorizations", and retrieves the appropriate DNS challenges from the acme server.

The next task will be to create a function that is able to verify the DNS propagation, and complete the challenges with Let's encrypt.

Static code analysis

Based on our discussions with the team, I also added some additional eslint rules and reformatted the code to match

Being the Sheriff

I worked through a lot of PRs, and wrote a bunch of reviews

by Denes Adam Dolhay at Sat Feb 04 2023 23:58:33 GMT+0000 (Coordinated Universal Time)

Week 3, Let's encrypt

This week I have dug a little deeper in the let's encrypt and node-acme-client.

Based on the research, there is two ways how node-acme-client is able to work. In manual and in automatic mode.

Because of the task queue system that we are planning to implement, it became clear, that the optimal solution is to use the manual mode, and create separate tasks for the following stages

  • Creating an order with let's encrypt, get all challenge data
  • Wait for DNS propagation, request challenge verification of individual challenges (reschedule until all done)
  • create the CSR and finish the certificate generation process

... More to come in the following weeks

by Denes Adam Dolhay at Sat Feb 04 2023 23:39:55 GMT+0000 (Coordinated Universal Time)


Eakam

Starchart: Playwright and Chakra UI Setup

The past week I worked on setting up Playwright, and Chakra UI with starchart. Chakra UI is needed so that their components could be used to build the web pages, and Playwright is to be used for end-to-end testing.

Chakra UI Setup

Setting up Chakra UI was relatively simple. I followed their instructions for the most part. I also found this example to be really helpful. My process for this went like this: I looked at the instructions in Chakra UI docs for the client, server and root files, and compared them with the existing code in starchart. Then, I also referenced them against the example code.

Not an ideal process to follow, but since I do not understand Chakra UI or Remix that well yet, I felt that this was the best way to do the setup. It worked, and I created a PR. Based on some feedback, I added some basic components (instead of using html tags) to the index page so some manual testing could be done.

Now, if you followed the PR link, you would have seen that it was merged but had a failing check. And that is what was addressed later while adding Playwright. Since the project was setup using the Remix blues stack, it shipped with cypress for end-to-end testing. The cypress folder had some tests but these were not modified when the app was setup since we intended to use Playwright. This resulted in the typecheck job failing, since it was also checking the cypress folder: tsc && tsc cypress

Playwright Setup

Setting up Playwright took longer than I had expected. First, I made a list of node modules that needed to be removed. These included cypress, @faker-js/faker, @testing-library/cypress, and eslint-plugin-cypress. These were only used by cypress so I uninstalled them using npm uninstall.

Next, I deleted the cypress folder, and the configuration file for cypress (cypress.config.ts). I installed playwright by using npm init playwright@latest, and following the prompts (docs). This also generated a config file for playwright, some simple tests in the specified test folder, and some more detailed examples of tests. I deleted the folder containing the examples since it was simply for reference, and added some basic tests.

For the configuration, I ended up using the defaults for most of the options. I also uncommented the configuration for mobile viewpoints, branded browsers, and output directory for test artifacts; and added the following based on some feedback:

baseURL: `http://localhost:${process.env.PORT}`
...
video: 'on-first-retry'

Setting a baseURl allows navigating to a page with a relative URL:

await page.goto('/');

The second configuration records a video of the test on the first retry after it fails.

Trying to Setup CI

So far, this process was pretty simple. Installing playwright also auto generates a GitHub workflow file. I tried adding a new job based on this auto generated file, and updated some npm scripts that were previously being used for cypress to run the tests. However, to run the tests, the app must be started. And to do that, the mysql database container must be running. I tried using the defined docker script, which uses docker-compose up -d to start the container but prisma db push failed with the following error:

I tried a bunch of things to try and fix this. I posted about this in Slack, and found out that mysql can take a while to start. So, I tried adding a wait before running the tests based on this discussion. However, this still resulted in the same error. I even tried doing prisma db pull before prisma db push. However, I got a different error this time:

This was really strange, since the database should have been setup. Still I did not know why this was failing.

Using Service Containers

Since I was basically out of ideas on how to fix this, I tried completely reworking the job by using a service container for mysql instead:

  E2E-Test:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:8
        ports:
          - 3306:3306
        env:
          MYSQL_DATABASE: starchart
          MYSQL_USER: starchart
          MYSQL_PASSWORD: starchart_password
          MYSQL_ROOT_PASSWORD: root_password

    steps:
    ...

And the tests worked! I am still not sure why they were failing before. Maybe docker-compose doesn't work with GitHub actions?
With this, I marked the PR ready for review, and the broken CI was fixed.

by Eakam at Sat Feb 04 2023 04:38:53 GMT+0000 (Coordinated Universal Time)


Tymur Levtsun

Starchart: Designing And Theming

Overview

This week I put my main effort in terms working on StartChart in two things:

  • UI Design (and collaborating on design with Mario)
  • Initialising Theme for the Chakra UI

I also did some code review, but they were side quests.

Designing

This week I kept working on UI design with Mario. We presented our design concept of Domain management screens and had very productive conversation with out team members. They provided us many clarifications on how system is gonna work and we adjusted the design accordingly. For now design of domains management screens look very decent.

Chakra UI init

The one PR I created this week was initialisation of Chakra UI theme. For now it only includes shades of our "brand" color, which is well-known Seneca's red color. This color is now applied as a default one for all UI components such as Button, Checkbox etc. After several iterations of code reviews and updates from my side the PR was merged.

Some thoughts

Hope the next week I will flex with some actual view elements be merged :) Have great weekends everyone

by Tymur Levtsun at Sat Feb 04 2023 04:17:03 GMT+0000 (Coordinated Universal Time)


Chen-Yuan Chu

Startchart: Prisma Schema

Background

This week, I focus on creating an initial schema for the project through using Prisma. This is for a MySQL database, and the PR for the initial schema is here.

This is an example of a model:

A model describes how a table would look like on a database.

Basics

  • Record would be the table name.
  • Green texts are the value types for the columns.
  • The blue texts are the names of the columns.
  • @id marks the primary key.
  • ? marks a column as nullable.
  • @default sets the default value as what's in the parameter.
  • now() returns the current time.
  • @updatedAt would provide the value that's the time a row is updated.
  • Foreign key:
    In the example, the line with user is used to record a relationship between this and the other table, for the purpose of marking a foreign key. fields's value is the foreign key, while references's value points to a column on another table.
    On the other table, it need to have another corresponding line to fully establish the relationship, like:
    [] means it's a to-many relationship. You don't have that if it's to-one relationship.

  • You might notice some odd type like RecordType. That's enumeration, and you can declare that in the same file:
    As you can see, the green text is the name of the type, and the blue texts are the possible values for such type.

Experience

As far as creating schema goes, Prisma is very friendly and easy to use, once you learn the basic, such as how to mark primary key or make values in a column unique. Establishing foreign key and the relationship between tables take a bit more work to understand. It helps if you already know how a database works.

Because I need to create the database to serve the need of other parts of the project, such as what user information people who handle login want to store, I ask around the team. I sort of feel like someone who goes around with a clip board and ask what they would like to order for lunch. I ask them what they would like to include and why...to have better comprehension of the project and for my own learning, and it's always interesting to learn more.

by Chen-Yuan Chu at Sat Feb 04 2023 02:40:41 GMT+0000 (Coordinated Universal Time)

Friday, February 3, 2023


Ririio

Rebase is confusing

It was engraved into my mind the importance of doing a rebase. Having all commits compiled together into a singular one sounds amazing, and clean when sending out a pr.

Rebasing didn't sound bad, the steps were actually quite simple

  1. Push a commit everytime you completed a section of your overall task
  2. Once all is done, do a rebase interactive to main from your working branch and squash all of them together
  3. Checkout to your working branch and push it

The Problem

The problem occurs when someone reviews the code, and I wanted to pass my newly added code. The rebase interactive doesn't seem to contain anymore values. That's a problem, because if I'm going to be pushing my updated code, I will now have two commit as to one. It doesn't sound like that big of a deal, but if you have to push a code every time someone does a review, you'll end up with several to up to a dozen commits on your pull request like this pr did.

How to solve the issue

The steps I told at the start was not all wrong, but if you noticed one thing, I actually never do a pull upstream. What do you think would happen if I pass a code that is behind the main branch? I would have conflicts. If I simply keep on doing what I did with rebase, I will consistently have conflicts. Now you're asking, what are the actual steps onto doing a proper rebase? the same steps apply

  1. Push a commit every time you completed a section of your overall task
  2. Once all is done, do a rebase interactive to main from your working branch and squash all of them together git rebase main -i

Now you add extra steps...

  1. Do a checkout to the main branch git checkout main
  2. Update your main branch to its latest instance through git pull upstream main
  3. Checkout back to your working branch

Now instead of pushing it, you do another rebase but there's no need for interactive, because you only have a single commit

  1. Remember to do a force push, because you are trying to alter a previous branch history git push -f origin main

Always make sure that before you do anything, that your main branch is always clean. If you are not sure simply do a git checkout -B main upstream/main, this should reset your main to the latest commit from the original repo.

Remember that upstream is only for fork repo, you should be using origin from your own

by Ririio at Fri Feb 03 2023 23:10:38 GMT+0000 (Coordinated Universal Time)


Wonkeun No

Lesson Learned From Code Review

I had rounds of code reviews for my PR to add Route53 DNS module to Starchart. Here's what I learned from the code review.

Working code is not enough, it should be easily readable for other developers

I wrote a createRecord function that creates a record in Route53 hosted zone. At the beginning, it didn't have hostedZoneId as I was thinking we would use only one hosted zone and I could treat it as an environment variable. Then I needed ability to set a hosted zone for testing purpose and added hostedZoneId to the function as the last parameter. But this is not a good practice. Some people would think hostedZoneId is something that exists inside a record. It works same if I add hostedZoneId in any order in the function, but the code should look clearer for others to understand.

Don't let same logic duplicated on the code

Even though I know this rule and try not to write same logic in different functions, but it occurs when I don't look at my working code closely again. Duplicate code can be reduced by reviewing and refactoring multiple times before pushing it to repo. Again working code is not enough. Repeating code will result in bugs once someone fails to modify all of the same logic. Good refactoring skills come after a lot of practices. Once you confirm that the code is working, it's time to find places to improve.

Function should do only one thing whenever possible

This is a good principle to write good tests or write tests more easily. Also it's good to produce the right error message when something breaks. Function name should be as clear as possible. validate() does not give any clue what to validate. If it can be broken down to validateRecord() and validateDomain(), it will be much easier to read and probably people don't have to look into function implementation.

Don't use unnecessary syntax and use linter to set a rule to maintain consistency

Which code looks cleaner and easier to read?

function foo() {
    if (x) {
        return y;
    } else {
        return z;
    }
}
function foo() {
    if (x) {
        return y;
    }
    return z;
}

Both work same, but we don't want to write unnecessary statement. We can avoid unnecessary syntax or error-prone code by using linter. It will help you get used to certain principles.

Conclusion

You will look much more professional if you look through your code and improve before someone reviews. Also thoroughly take reviewer's comments and discuss if you have a different opinion. Opinions are always debatable. You will learn more from discussion.

by Wonkeun No at Fri Feb 03 2023 16:46:26 GMT+0000 (Coordinated Universal Time)

Monday, January 30, 2023


Stefan Frunza

Breaking Telescope

A 400 error

It all starts with an old 400 error that was occurring in Telescope for a while now, first documented here.

I had found that when we do the SAML sign in flow for authentication we seem to have, on occasion, a blocked session id cookie on the callback when the SAML response returns to us.

I could not reproduce this in the local setup, so I resolved to push tests for possible fixes to prod and David let me, which I am sure he regrets now.

This is the starting point for my experimentation and inevitable breaking of Telescope prod.

The first idea

This is my first PR and it outlines my first ideas.

Directly from the PR:

There is a 400 we sometimes get because the cookie that is sent to us with the post binding from Azure AD is sometimes blocked since it is from a different origin/site. Enabling sameSite 'none' should hopefully fix this. By default sameSite is set to 'Lax' which sometimes block incoming cookies from other sites such as the request AD makes to our callback.

This is all outlined nicely in the following document David found https://web.dev/samesite-cookie-recipes/#unsafe-requests-across-sites.

This pattern is used for sites that may redirect the user out to a remote service to perform some operation before returning, for example redirecting to a third-party identity provider. Before the user leaves the site, a cookie is set containing a single use token with the expectation that this token can be checked on the returning request to mitigate Cross Site Request Forgery (CSRF) attacks. If that returning request comes via POST then it will be necessary to mark the cookies as SameSite=None; Secure.

This took Telescope from sometimes not working and returning the 400 to ALWAYS not working and returning a 400. Luckily I am not out of ideas, but unfortunately none of them have worked since either...

Second idea

My second PR. Since in production Telescope express server is behind two proxies in Traefik and nginx I thought maybe using this trust setting with 2 hops would be enough to allow the requests to maintain their cookies even now that they were set as secure and sameSite none. This was not the case.

Example:

var app = express()
app.set('trust proxy', 1) // trust first proxy
app.use(session({
  secret: 'keyboard cat',
  resave: false,
  saveUninitialized: true,
  cookie: { secure: true }
}))

My final ideas

My final ideas before resetting prod that I have not yet tested are to change the origin to be explicit instead of a wildcard *. Because I have read that cors needs to be specified if the cookie is secure. I also want to try setting express.urlencoded({ extended: true }) to false in Satellite src because in all examples I have seen so far they have it set to false which uses a different query string library and the default setting of true has been deprecated as per the docs. The odds that that could actually be the problem is really low but it's worth a shot.

Problem now is that the explicit origin is different based on the environment Telescope is run in, and I have to find a way to dynamically change it based on where/how it's being run.

Hopefully in any case I will be able to finish my testing and have it running at least as well as it was before I messed with it.

by Stefan Frunza at Mon Jan 30 2023 16:23:47 GMT+0000 (Coordinated Universal Time)

Saturday, January 28, 2023


Eakam

Starchart: release 0.1 planning

This week, after some discussion, a plan was made for release 0.1 of starchart. This involved getting an initial app setup using Remix's blues stack so that people could code on top of it. Next step involved creating some issues which could be addressed in release 0.1 release 0.2 and some for further releases. As I am mostly focused on front-end and testing, I created the following issues:

The first three issues are the ones that I plan to address in the next few days for release 0.1, which would be next Friday. These could be done in any order but I think setting up Chakra UI with Remix should come first since this would allow the use of Chakra UI components for building the pages.

This could be followed by adding jest and playwright for testing. Part of this also involves removing any unneeded dependencies that the Remix blues stack comes with, such as Cypress, which is another end-to-end testing framework.
Adding a healthcheck route could also be done in release 0.1 or release 0.2. This would be a /healthcheck route that returns a 200 response with a message stating that the server is running.

The other issues would take longer to address. The user domain management page is supposed to be a page which allows users to view and manage their existing domains. This page also links to the domain creation page, which has a form to allow creation of new domains.

The admin dashboard would serve as a place for admins to view all the certificates, domains, and users. This also includes the ability to search and filter through this data, and update any records. Some aspects of the user dashboard could be reused for this page as well.

Adding these pages requires a design to be created first. One such issue is being worked on to create designs for user dashboard pages using figma (#36 Create user dashboard pages design).

by Eakam at Sat Jan 28 2023 04:58:08 GMT+0000 (Coordinated Universal Time)


Chen-Yuan Chu

Starchart: Planning Toward Release 0.1

This week, we some planning for Starchart release 0.1.
The first waves of issues were filed, and tasks were assigned to the team members. I'm to contribute in setting up database schema and the ORM, Prisma, as well as finding out how to run it in Docker.

My Issues

I filed 7 issues in total:
Issue #6 is about adding a wiki entry for Prisma to serve as a start point for helping team members looking into the technology.
Issue #9 is about notifying users about their SSL certificate request status via email
Issue #10 centers around creating a page for user to view information of SSL certificate they have requested.
Issue #11 talks about a feature to notify users about their SSL certificate expiration and next-steps
Issue #12 is regarding documentation for helping developers setup their local environment for development of this project.
Issue #13 is regarding documentation for helping developers follow flow of contribution to contribute to the project.
Issue #16 is for creating a certificate table in the database to record information of SSL certificate created.

Technology Research

Because much of the tech stack is new to me, I have to spend time reading the documentations to familiarize myself with it.
Since I try to focus on database, my primary focus is on MySQL and Prisma, which is the primary way the project uses to interact with a MySQL database.
In my research, I found information for how to set up Prisma for a MySQL database, and I have tried the steps.
Through commands, one can run a .prisma file to push/pull table schema to/from the target database. One can also run commands to run a file with code that essentially act like SQL statements.

Here are some basic CLI commands:

  • npx prisma db pull to pull database tables into your schema file.
  • npx prisma generate to sync client to current schema in the local schema file; do this before you do CRUD SQL operation via prisma.
  • npx ts-node <file location> to execute CRUD operation code in the file.
  • npx prisma migrate dev --name <any migration name> to migrate your schema.prisma file to target database. Because prisma make a shadow database (temporary) every time to make sure database is sync, use npx prisma database push if you don’t have right to make another database (no benefit from shadow database)
  • npx prisma studio to start Prisma studio, the visual editor UI for database.

Also, I tried to look up how to run MySQL database in Docker. It seems you need to download an image of a MySQL database, and then create an instance in Docker, as detailed on MySQL's website.
The Docker documentation suggests an one-line command to do so:docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag.

by Chen-Yuan Chu at Sat Jan 28 2023 03:49:52 GMT+0000 (Coordinated Universal Time)

Friday, January 27, 2023


Taimoor Dawami

Small Contributions to Starchart

This week, I set up the initial Starchart project using Remix's Blues Stack template. The PR is the starting point for adding, modifying, and removing features and functionalities. There were some problems running the development environment, as the user had to copy the default Postgres credentials from .env.example to a new .env file. I most likely forgot about including it as a step in the "Steps to Test" section of my PR because the .env file was created by the script I used to generate the template for the project. However, as we do not intend to use Postgres to store our data, and the purpose of the PR was to provide a starter code for the team, the PR was successfully squashed and merged.

I also got the opportunity to review a PR in the Starchart repo that dealt with adding package-log.json to the tree and removing dependencies with security issues. The package-log.json was automatically added to .gitignore when I ran the script to generate Blues Stack project, so I pointed out that we should add it back to the tree as it's an essential file for several reasons. One is that it has an optimization purpose of letting npm skip previously installed dependencies during the installation process. Another thing worth pointing out was removing prom-client since it is a Prometheus-related dependency we are trying to avoid.

Some of the errors from my end could have been avoided had I not felt the pressure of keeping other teammates waiting, as all the core features and functionalities rely on the initial Remix Blues Stack setup.

by Taimoor Dawami at Fri Jan 27 2023 21:41:42 GMT+0000 (Coordinated Universal Time)


Tymur Levtsun

First contributions

Overview

This is the first week when I finally contributed something to the project and wasn't just digging into some technology. On top of that, I also did some code reviewing (which I guess is also a contribution) and tried to get more familiar with some other technologies that we are going to use.

The week 3 contributions

UI design

The first thing that I did this week for the project and one that took the most effort was creating some initial design for the application. I am not a UI/UX designer myself, but I worked a lot with these guys and I am kinda familiar with their tooling. The way to go for me was Figma since Charka UI (the UI component library that we were going to use) has the Figma template and I used it before. On top of that, Figma has incredible functionality in terms of collaboration and since this is an open-source project, I found this characteristic very important.

I am not the super creative guy, so I ended up with a design that looks kind of plain, but it's way better than having nothing. At least, Since I am a front-end developer myself I know that this design is gonna be easy to implement in code.
Link for the design: https://www.figma.com/file/XAcnMIC1y4qumWinSalHI9/Starchart-UI?node-id=2105%3A2109&t=w4LYaSLX7CLNLfT8-1

Code reviews

We didn't have a lot of pull requests this week, so not many of them to review. It was one, where TD initialized remix project with the Remix Blues Stack template. The second PR I checked was David's, where he removed express-prometheus-middleware since we are not going to use Prometheus and he also did some minor package.json related changes.

by Tymur Levtsun at Fri Jan 27 2023 19:36:30 GMT+0000 (Coordinated Universal Time)


Ririio

Project Planning

This week we had a meeting regarding the implementation of Release 0.1 of our project starchart. There was a lot of information being thrown around regarding features we have to implement before the initial release. At the end of the discussion professor had asks around regarding features we'll be implementing for the initial release, and I was provided with the design template, and managing of eslint, prettier, and typescript configuration.

TypeScript

I have actually never written anything from typescript before. I did contribute in some ts repositories during hacktoberfest, but my knowledge only came from reading the code, and ensuring I implement them as closely to the original author as possible.

Ever since wednesday I've been watching videos, and doing my own research regarding typescript, and what's the best settings for its config file

Chakra UI

I've made a few progress with chakra this week. I have another project that I'm currently helping build for one of my courses, and I convinced my members to use the library for the front-end. It also helps with having a deeper understanding with the library so I can create as clean of a UI as I can.

Design Template

I was working with Tymur regarding the templates that we'll be using for our dns web application. The template are bearing similarities to blackboard when it comes to the use of theme and styling. I believe that it's for the best to have the website have some similarities with the official seneca website to ensure that students who wants to borrow a domain feel some sense of familiarity with it.

Progress

For the rest of the week until Friday of next week, I'll be working with finding the typescript configuration settings, and customizing both prettier and linter to follow good coding conventions

by Ririio at Fri Jan 27 2023 04:58:47 GMT+0000 (Coordinated Universal Time)

Thursday, January 26, 2023


Wonkeun No

Create Domains Using Route 53 SDK

The team for Starchart project is in planning phase. It's not as exciting as coding. But research is important and it's good time to explore technologies that I have not tried or learn more details about things that I tried. Starchart will create a domain for Seneca students. To do so, the team will use Route 53 and I spent some time on understanding AWS SDK for Route 53 since we need to automate the steps to create a domain in Starchart web app.

Setup Route 53 SDK

To learn Route 53 SDK, I started with official SDK page. To use Route 53 SDK, you need to install SDK by running npm install @aws-sdk/client-route-53.

Now I need to start coding. First step is import Route53Client and set up configuration.

const { Route53Client } = require('@aws-sdk/client-route-53');

const config = {
    credentials: {
        accessKeyId: '******',
        secretAccessKey: '******',
        sessionToken: '******',
    },
};

You need to get your own credential information from your AWS account. I use student account which might be different from normal users. I can get my accessKeyId, secretAccessKey, and sessionToken in AWS Details as shown in below picture.

Creating Hosted Zone

The first method that I need to try is CreateHostedZoneCommand. Hosted zone is a AWS term that represents a collection of records that can be managed together, belonging to single parent domain name. And here's code snippet to use CreateHostedZoneCommand.

const { Route53Client, CreateHostedZoneCommand } = require('@aws-sdk/client-route-53');

const input = {
        CallerReference: new Date().toString(),
        Name: 'test.starchart.com',
    };

const client = new Route53Client(config);
const command = new CreateHostedZoneCommand(input);
const response = await client.send(command); 

To create a hosted zone, all you need is CallerReference and Name. You can also set up more input parameters such as HostedZoneConfig, VPC, etc. based on your need. CallerReference must a unique value for your request. If you use the same CallerReference, you will get an error. And Name is for your domain name. If response is successful, you will receive metadata of your request, location, change info, hosted zone info, nameserver info.

Now I have records inside the hosted zone that I created.

Add/Update/Delete Records in Hosted Zone

I can add a new record or update existing records with ChangeResourceRecordsSetCommand function. Below is the basic syntax how to create or update your record.

const { Route53Client, ChangeResourceRecordSetsCommand } = require('@aws-sdk/client-route-53');

const changeBatch = {
    Action: 'UPSERT',
    ResourceRecordSet: {
        Name: 'mydomain.starchart.com',
        Type: 'A', 
        TTL: 60,
        ResourceRecords: [
            {
                Value: '192.168.0.1',
            },
        ],
    },
};

const input = {
    ChangeBatch: changeBatch,
    HostedZoneId: '****************',
};

const client = new Route53Client(config);
const command = new ChangeResourceRecordSetsCommand(input);
const response = await client.send(command); 

For input parameter, you need to provide changeBatch object and HostedZoneId. In this case, I use UPSERT action. This is a way to simplify the method that it can add a new record if it doesn't exist or update existing one. In the ResourceRecordSet, you should provide domain name, type, TTL (i.e. record cache time to live), and route traffic to. In this example, the new domain mydomain.starchart.com will be directed to A record, 192.168.0.1. You can find more options when you set routing policy other than this simple example. I found AWS CLI Command Reference page is very helpful to understand the parameters and refer to other ChangeBatch syntax.

List Records and View Change Status

When you fetch all your records or partial records, you can use ListResourceRecordSetsCommand.

Lastly GetChangeCommand method can check the status of specific ChangeBatch by providing ChangeInfo.Id from ChangeResourceRecordSetsCommand response as a parameter. GetChangeCommand returns ChangeInfo and ChangeInfo.Status will show either PENDING or INSYNC.

Conclusion

I used AWS S3, DynamoDB SDK before. So it didn't take much time to set up and configure. AWS SDK syntax is highly consistent and it's easy to use once you learn any one of these SDKs. As I mentioned above, AWS CLI reference page has more details explaining each parameter and response although you will rely on JavaScript SDK page more to write your code. Now I'm eager to write code for Starchart.

by Wonkeun No at Thu Jan 26 2023 01:00:20 GMT+0000 (Coordinated Universal Time)

Wednesday, January 25, 2023


Ray Gervais

Technical Interests for 2023

Ray, what are your goals? What are your aspirations for 2023? Where do you want to take your career next? How’s the music coming along?

I cannot express the number of times I’ve been asked questions like these, and sadly I never could give a good answer to them, it was simply something that I didn’t have the mental capacity or energy to think of at the time. Over the Christmas holidays, whereas I should have started thinking about exactly these topics, I opted instead to have a lazy, non-future-forward-looking holiday; I delegated the thought experiment to be January’s problem. Yet, time, though linear, doesn’t feel to be linear and so, we’re nearing February. So, let me share with you some of my thoughts and interests that as of this moment, I want to pursue and explore this year when possible. Wait, what does that last part mean? Well. I’m familiar with burnout, and even with these interests, I don’t wish to dance the same dance I’ve done before which flirts so well with burnout, so I’m not holding myself strictly to the following interests as a must achieve, but more so as possibilities. As a metaphor, if given the keys to a candy shop, and you’re in the mood for candy, here are the candy that I’d select depending on the day. Terrible metaphor, I know. I hate candy, but you get the point I hope.

Programming Languages

Rust

I truly enjoy writing little CLI applications and other small items which run on a system, so the interest in such a language shouldn’t be a surprise. Furthermore, I want to build a few projects using it’s ecosystem so that I can attempt to understand all the developer hype which no one can escape from; plus, I have my own gripes with Go and want to compare the two. I’m not blind to the difference in common use-cases between the two languages, but when I’m feeling brave enough, perhaps I’ll experiment with melding my Cloud Native practice with an up and coming concept, https://github.com/awesome-rust-cloud-native/awesome-rust-cloud-native! Lastly, I want to revisit a practice of learning which I haven’t utilized since college, books! Specifically, 2022 release: the Zero to Production in Rust which I received as a gift over the holidays.

In some ways, I suppose I also have my friend Tryton to both thank and blame for the growing interest. These past few years have had us sharing interesting Linux tools & applications, comparisons between platforms, and evaluating how you’d write an application in Go vs Rust.

Kotlin

In my professional career, I’ve had the interesting circumstance where there’s never been a requirement which required me to work with a JVM language, not even a Springboot API yet. Having worked quite deeply with Go for the past two years, along with switching from iPhone to Pixel purely to see how the Android world has evolved, why not see how the developer experience and process has evolved as well? You may remember my blog post My First Impressions of Udacity’s Kotlin for Android Development, where I went through the available three lessons course provided by Google and was left wishing that they had released all of the lessons listed back in 2019. Upon reviewing earlier in 2022, I noticed that more lessons where added! Even-more-so, it appears that Google’s been going absolutely crazy partnering with various platforms to release a multitude of training and courses:

So, I have my eyes on completing the original course I had started, along with trying out Jetpack Compose for Android Developers this year!

Elixir

As you know from the occasional blog post, I’ve been diving into more and more functional programming concepts in my everyday work, and distracting myself with the excuse of learning “better ways to program” as I read various articles and tutorials at 2am. Haskell, though incredible according to the majority, seems too out of reach for me just yet, and Scala or Clojure don’t fit into my JVM curiosity while Kotlin is still the plan. Perhaps in the future if Kotlin doesn’t pan out, I’ll explore Scala or Clojure, but for now, I want to explore a language which has a platform of it’s own. Elixir fits that bill, and I can’t help but be intrigued at what it’s capable of. Plus, the Pipe |> operator language feature looks damn cool, and I’m scared that if I were to learn and use it, I’ll miss it everywhere else I go. I only have two concerns, but they don’t detract from the overall curiosity:

  1. Elixir is a dynamic language, which isn’t a problem by any means, but goes against my current preferences to write with statically typed languages where possible.
  2. Elixir is heavily influenced by Ruby, and thus uses Ruby syntax, which I have never really enjoyed. I know, call me the odd one out.

The Cloud Native Computing Foundation

In the past year, I’ve had the opportunity to dive deeply into working with a few projects from the CNCF landscape. Notably:

  • Argo, Flux: Continuous Integration & Delivery
  • Helm: Application Definition & Image Build
  • Operator Framework, Kubebuilder: Application Definition & Image
  • Tigera: Security & Compliance

And yet, there’s so much more within the CNCF! I could be naive and say that I want to learn them all, but that’s incredibly unrealistic and equally less possible without going absolutely insane. Instead, I want to approach what I believe to be a much saner and reasonable goal: test out and leverage a service from each category (of interest) in some project of my own. This way, I enable myself to be exposed more so to the cloud native concepts which elude me day-in-day-out while also equipping my understanding of how all the pieces fit together in smaller, targeted pieces. For those who see the CNCF landscape as an absolute mess, I’d highly recommend giving Navigating the CNCF Landscape, The Right Way a watch. Though the following could change, and I’m certainly just listing projects which seem appealing here, not so much implying they’ll be used in my own project and research, here’s where my gaze is dwelling:

Resources

by Ray Gervais at Wed Jan 25 2023 00:00:00 GMT+0000 (Coordinated Universal Time)

Monday, January 23, 2023


Wonkeun No

How to Write Dockerfile and Basic Docker Commands

Seneca OSD700 team project Starchart will use many Docker containers. I would like to summarize how to write Dockerfile and basic Docker commands.

First we need to write a Dockerfile which is a set of instruction to build a Docker image. A Docker image is a read-only template with instructions for creating a Docker container and the container is a runnable instance of an image. It's probably better to understand with below node project example.

FROM node:16.15.1
LABEL maintainer="Wonkeun No <wonkeun.no@gmail.com>"
ENV PORT=8080
WORKDIR /app
COPY package.json ./
RUN npm install
COPY ./src ./src
CMD npm start
EXPOSE 8080

Dockerfile must begin with FROM to specify what parent image the container uses. In this case, I use node version 16.15.1. So my app will be running on top of node image which someone built for everybody.

LABEL is not a mandatory command, but you can add some metadata information about the image as a key=value pair. You can add more like version, description, etc.

Environment variable can be set with ENV command. It can be overridden with docker command.

WORKDIR sets the working directory. Multiple WORKDIR is also possible. When a relative path is provided, it will be relative to previous WORKDIR.

Then, copy package.json and paste it to ./ directory with COPY instruction. And run npm install with RUN to download npm dependencies in this case.

Finally copy all ./src files and paste into container's ./src directory and run npm start with CMD with port 8080 open outside of container. You can run many RUN instructions, but CMD can be run once as a container launch command. You can still write multiple CMD, but all will be ignored by last CMD.

Once Dockerfile is written, it's highly recommended to ignore some of files that are not necessary in docker build by adding them to .dockerignore. This process helps to reduce the size of image and build faster. You secret information files should not be in the image either.

There is more information about Dockerfile in the official Docker site. Docker reference

Now it's ready to build the image with below command.

docker build -t <image>:latest .

-t can specify a tag for the image. This option can be combined with -i (i.e. shortened form of --interactive) to interact with container in a terminal. It's useful to see what is going on inside the container.

If the build is successful, you can find the image with below command.

docker image ls 

Now it's time to actually run the container.

docker run --rm --name <image> --env-file .env -e LOG_LEVEL=debug -p 8080:8080 <image>:latest -d

Here's explanation for each option.

  • --rm removes the container when it exits automatically
  • --name assign a name to the container
  • --env-file specifies a file that defines environment variables
  • -e sets an environment variable for the container
  • -p binds the host machine's port with container's port (i.e. [HOST_MACHINE_PORT:CONTAINER_PORT]
  • -d runs the container in background and print container ID

Once you run above command, you will see the container's ID. Run below to see the logs from the container. -f option is to continuously print out the logs as it generates.

docker logs -f <container_id>

This article helps to create a Docker image and run a single container. I will write more about best practises of writing Dockerfile , DockerHub, and docker-compose.

by Wonkeun No at Mon Jan 23 2023 20:54:31 GMT+0000 (Coordinated Universal Time)


Tymur Levtsun

Choosing my way in Starchart

Overview

This week I had to choose my role in Starchart, and what part of the project I want to work on. I find front-end development using Remix, React, and Charka UI as the best field where I could apply my skills and competence.

Background

I have pretty strong front-end/mobile development experience as a student. I have been working on commercial projects for the last 4 years where I was using majorly React, Next.js, React Native and many other side technologies.

Why I find this interesting

The Remix is something that I have never worked with before, even tho I have seen some videos about its potential and power. I have worked with Next.js before which is very close to Remix in terms of what problem it resolves. I see Remix as a technology that might become a replacement for the Next.js if maintainer develops it properly.

Typescript

Another thing that made me excited about the Startchart development is the fact that it is going to use Typescript, instead of Javascript. I have been using typescript for last year in all projects that I worked on and it is just so much better to have static types in your code, instead of getting numerous undefined exceptions when developing with javascript.

Conclusion

I will be happy to work on the Start chat front-end side, even tho I want to participate in other parts of the project to, at least understand how these technologies work on the basic level.

by Tymur Levtsun at Mon Jan 23 2023 07:18:27 GMT+0000 (Coordinated Universal Time)

Sunday, January 22, 2023


Ririio

Starchart: Choosing a technology

Finding a path

By the end of last week, we were informed that for the creation of our project (Starchart), we will be using a specific technologies for it. Such technologies are...

I have my career planned out for a while now, and I planned on working with the front-end to get accustomed with the work, and also because it has a visual presentation of what my code is doing in real time (I'll work my back-end once I'm more accustomed). Thus, when choosing a technology to work on, I decided to pick Chakra UI, due to it being a library of react components (which I'm a bit familiar of). Having a thorough understanding of the library should help me have a much deeper understanding of the ReactJS library.

What I found

The reading I did made me realize that Chakra UI actually helps keep a group much more organized when working with a project. The problems I noticed with working with other people, is that individuals have different answers for similar questions. If you tell someone to create a side navigation bar, what they'll come up with will be quite different from yours. I'm not referring to the design, but the source code. With the use of pre-made components, chakra ui reduce the burden of having to figure out your peer's work, and help new contributors get in to action much faster.

I'm planning on implementing this library to a project I have for a different course. Working with people with no experience will bound to be un-organized, and this should help lessen that issue.

I'll be updating the wiki once a week to help contributors to the project a basic knowledge of what chakra is

by Ririio at Sun Jan 22 2023 15:03:38 GMT+0000 (Coordinated Universal Time)

Saturday, January 21, 2023


Eakam

Starchart: Picking a project area

As we continue to work on developing starchart, which is a project that aims to make it easier to Seneca developers to create SSL certificates (without worrying about the costs or the need to provide personally identifiable information), a list of task groups has been created to make it easier to visualize the different parts of the project. These include Web, Testing, Server, DNS, etc. (full list here).

I have spent the last week thinking about where I can contribute. An obvious choice was Testing as I spent the previous week researching Playwright, which is a framework for end-to-end testing. As I looked into some of its features, I found that it has auto-wait functionality for its tests by default. Basically, before an action is performed on an element(e.g. click), it will wait for it to be in a specific state (e.g. attached to the DOM) based on the action. This reduces flaky tests which fail if a statement executes before the element is ready.

Further, through Codegen, tests can be automatically generated based on actions performed in the browser. This should reduce the time required for writing tests quite a bit. So, in addition to working on testing, I also decided to contribute to the Web component, which involves building the pages for the app. This will be done using Chakra UI and Remix. Chakra UI provides a bunch of components for different needs such as tooltips, cards, popups, and many more. I have worked on building webpages a lot in the past five months, and even though I have never used Chakra UI, I think I would still be able to utilize some of that experience here.

Finally, this is meant as a guideline, and not restrictions, for which parts of the project I would like to contribute to.

by Eakam at Sat Jan 21 2023 05:02:10 GMT+0000 (Coordinated Universal Time)


Stefan Frunza

Choosing a path

As I look to the semester ahead I am petrified. Part of it is that I have to get used to Telescope and even just getting it running locally takes a while since I have to start WSL, Docker and the containers, that's before actually learning how to work with it or getting started trying to navigate testing or debugging or anything like that, the other part is that I really don't know that much in general but that's ok since I have a few places to start, although I guess that too is kind of overwhelming.

When looking at the starchart project, which I don't think will be called starchart but that's what we have been calling it so far, I have decided that I want to work on a bit of everything, to start I particularly want to take a look at something that is new to me and that I have bumbled my way through a short presentation on and that is authentication using the SAML protocol.

There are a few places to start with this, one is that we have an issue in Telescope that relates to what I think is the session no longer existing in certain situations that results in a 400. This will give me an opportunity to take a look at Telescopes auth flow. I still am kind of unfamiliar with Telescope despite having been in the last OSD course. I know that we use passport SAML in Telescope however and in the new project we will use a different library called SAMLIFY.

Another starting point is right there, Samlify docs. Further than that David found a perfect example for our tech stack where someone has made an example implementation of SAMLIFY with REMIX here. So taking a look at these things would be a start.

Also reading up on the rest of the tech stack would be smart as well. It's all overwhelming up-front but I think will become normal as we go.

by Stefan Frunza at Sat Jan 21 2023 00:58:36 GMT+0000 (Coordinated Universal Time)

Friday, January 20, 2023


Taimoor Dawami

Planning: Using Satellite features in Starchart

Overview

Starchart is a new project being put together by developers at Seneca College. The project will help students manage SSL certificates and subdomains without too much hassle and free of cost.

The project is currently in the planning phase, and being part of the development team, I have been researching ways to add relevant bits of Satellite in the Remix Blues Stack project, we intend on using for Starchart.

The Plan

After studying Satellite and Remix's Blues Stack source code, I compiled a list of features and functionalities we can port from Satellite. It's better to port features from Satellite rather than replace Remix's Blues Stack server with it, as Satellite comes pre-bundled with some dependencies that we do not require in Starchart.

Proposed Satellite Features

  • Logger
  • Middleware
  • Hash
  • Create Error

Proposed Dependencies used by Satellite

  • pino, pino-http, pino-pretty (to use Logger)
  • http-codes (to use Create Error)
  • Terminus (for health-check and/or shutdown)
  • helmet (for setting http headers)
  • express-jwt (for middleware)
  • cors

Proposed File Structure

After generating a Blues Stack project, we could create a directory named "Satellite" where all the files from Satellite would live. The structure will look similar to the Satellite directory within the Telescope, with only files relevant to Starchart. After installing the needed dependencies, we need to refactor the code to use Typescript and ESM (import/export) syntax to ensure it works.

Server.ts Changes

Since we are using Pino-based logger, we would import logger in server.ts and pass that as a middleware instead of passing morgan.

healthCheck

There's already a route setup for healthCheck in the app directory within Remix Blues Stack project, we could tweak that file to configure terminus using createTerminus function for healthCheck functionality.

Middleware

We could utilize the isAuthorized and isAuthenticated middleware functions to authorize and authenticate users. Also, we could benefit from the errorHandler middleware.

Hash

I am unsure if we need this function, as it's helpful for hashing information that is safe to pass in URLs. We cannot use it for hashing passwords, so if there's no need for hashing information in the URL, we do not need to port this function.

Conclusion

Once the team decides what features and functionalities we need from Satellite, we could start by filing an issue for it in the Starchart repository and go from there.

by Taimoor Dawami at Fri Jan 20 2023 22:03:45 GMT+0000 (Coordinated Universal Time)


Wonkeun No

Initiating a Team Project Creating Domains for Seneca Student

The team in Seneca OSD700 course has started a project called Starchart. This web app will create the number of custom domains for each student with a SSL certificate. More exciting part is that this web app will go live for all of Seneca students which is very rewarding. It will be a huge open-source experience to make something real from scratch. All of team members are very inspiring and I'm looking forward to working altogether.

We will use a bunch of new technologies for Starchart. One of the tech stacks that we will use is AWS Route 53.

Route 53 is a DNS service that sends client's request to the right resources. Basically Route 53 can provide 3 main services.

  • Register a name for the website
  • Connect the browser with the website or web application
  • Check health of resources by sending automated request to a resource

Like other AWS services, Route 53 is highly reliable and scalable, easy to use and cost effective, and secure. There are many Route 53 features. Some people are interested in configuring various routing policies and continuously monitoring resources to send requests to alternate routing. But in our use case, we need to automate the domain creation and find some ways to monitor these domains. More details can be found on this page.

We can use Route 53 in Management Console with graphical UI, also via API or SDK. Although we will use SDK, I would like to show how to create hosting zone in Management Console for someone who needs it and have an idea what SDK does. (Photos are taken from the passionate team member, Alex Romanova. Thank you!).

  1. First, go to Route 53 dashboard and click Create hosted zone.

  2. Add a domain name to create a hosted zone on Create hosted zone page. Then click Create hosted zone button on the bottom. (A hosted zone is a term for AWS that represents a collection of records that can be managed together, belonging to single parent domain name. All records within hosted zone must have the hosted zone's domain name as a suffix. In this case, www.subdomain.bytechlabs.com, but not www.subdomain.bytechlabs.ca.)

  3. You will see your hosted zone detail page with a list of records and nameservers, etc.

However, this is not what we need for Starchart. We have to automate this process using JavaScript SDK. To do this, I need to install AWS SDK package.

npm install aws-sdk

And this is the code snippet to create hosted zone with SDK.

var params = {
  CallerReference: 'STRING_VALUE', /* required */
  Name: 'STRING_VALUE', /* required */
  DelegationSetId: 'STRING_VALUE',
  HostedZoneConfig: {
    Comment: 'STRING_VALUE',
    PrivateZone: true || false
  },
  VPC: {
    VPCId: 'STRING_VALUE',
    VPCRegion: us-east-1 | us-east-2 | us-west-1 | us-west-2 | eu-west-1 | eu-west-2 | eu-west-3 | eu-central-1 | eu-central-2 | ap-east-1 | me-south-1 | us-gov-west-1 | us-gov-east-1 | us-iso-east-1 | us-iso-west-1 | us-isob-east-1 | me-central-1 | ap-southeast-1 | ap-southeast-2 | ap-southeast-3 | ap-south-1 | ap-south-2 | ap-northeast-1 | ap-northeast-2 | ap-northeast-3 | eu-north-1 | sa-east-1 | ca-central-1 | cn-north-1 | af-south-1 | eu-south-1 | eu-south-2
  }
};
route53.createHostedZone(params, function(err, data) {
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log(data);           // successful response
});

As you can see, CallerReference and Name are required parameters. Name is domain name as shown above graphical UI example. But what is CallerReference? CallerReference must be a unique string every time when running CreateHostedZone to avoid executing the operation twice. It's recommended to use something like date/time stamp.

There are much more methods and guidance on this page. We are not trying to provide full DNS services. I will write functions with methods that will create, update, and delete domain. And some of available domain monitoring methods. From next week we will be creating many issues and writing codes.

I've never done a team project from scratch. Everyone has a different interest and role. But we will have to help each other. So it's going to be very interesting and a lot of learning how we start building and coordinate the work together.

by Wonkeun No at Fri Jan 20 2023 19:16:19 GMT+0000 (Coordinated Universal Time)


Chen-Yuan Chu

Charting the Starchart

New Project

Recently, I’m participating in a project called Starchart. It’s going to be an easy way for Seneca developer community to obtain SSL certificates for their websites as they develop websites.
Unlike the usual complicated method, they simply need to fill a few boxes and click a button to get one.
This is a bran new project still in its planning stage, so there’s a lot of works to do. Because I’m interested in databases, I want to take on building that part of the project with others.

What do I do?

The tech stack we use would use Prisma to interact with a MySQL database for storing data we will be using. Because Seneca IT will handle upkeep of the database, we don’t need to worry about it. As Prisma is new to me, I have a lot of learning to do, and it should be an enlightening experience.

Since we need to keep track of certificate creation, we need a database table for information of certificates, like date pf creation. We also need some user information on a user table to store things like email. These are to help use email users when their certificates are about to expire. The table will need to be related via a foreign key. Perhaps something like user IDs. The primary key for the tables might be user IDs. We will need to discuss to hush out the detail before we build the database tables.

Another thing to figure out is how to have MySQL run in Docker container, for developers to work with during development. I think this is another thing I need to look into. I have used existing Docker setup through simple commands for other projects before, but I haven’t had any experience of setting it up myself.

What ELSE to do?

If I have the spare attention for other things, I suppose I can try to help tackle front-end of the project.
Having worked on other large scale projects’ front-end, such as HTML and CSS, makes me feel a bit more comfortable with doing so.

by Chen-Yuan Chu at Fri Jan 20 2023 01:13:37 GMT+0000 (Coordinated Universal Time)

Thursday, January 19, 2023


Denes Adam Dolhay

Week 2 - my part in the project

Like any good project, ours is also built on developers, with diverging interests and experiences. This fact is instrumental, as people with different knowledge and experience in various disciplines represent different viewpoints and contribute their own, unique key pieces.

About me

I started my career with building server hardware. As the business grew, we slowly graduated from towers through rackmount to blade server infrastructure, building out racks with hundreds of CPUs.

During this, I also got into Systems Architecture, DevOps and Networking as the company built out its own, custom made private cloud solution (LIR -> BGP routing, dedicated storage, worker nodes, virtualization, etc.)

Relevant experience:

  • Systems architecture
  • Git / Git workflow / CI/CD
  • Mysql
  • DNS
  • SSL + certificates
  • Computer networks
  • NodeJs
  • React
  • Shell scripting
  • End-to-end testing (Cypress)

What I'd like to work on

mostly the Let's encrypt part of the project.

by Denes Adam Dolhay at Thu Jan 19 2023 21:31:36 GMT+0000 (Coordinated Universal Time)


Tymur Levtsun

Telescope: How I set it up

Overview

This week my task was to setup the telescope running locally and tell to my classmates how it went, so here I am :)

Environment setup guide

As a good well-established project, Telescope provides a great environment setup guide, which covers most use cases.

How it went to me

Setting up Telescope locally was kinda a problem to me, since after I run all proper commands like:

  • installing pnpm globally
  • installing dependencies with it: pnmp install
  • run unit tests with pnpm test

I was having an issue with one dependency, so after several attempts to resolve it myself I reached the Telescope's community and filed an issue at the Telescope's github.

About 20 minutes after one of the maintainers reached to me and recommended me a way to resolve my issue that helped.

After that I was able to run the Telescope docker container with the following commands:

  • pnpm services:start
  • cp ./src/db/env.example .env
  • pnpm db:init
  • Removing the parser container (I just did it from the Docker UI)
  • Run parser with pnpm services:start parser

Everything seems to run fine now :)

What I learned

The interesting seen that I have never seen before is pnpm It looks like it is an npm on steroids. As far as I see it does pretty much what npm does, but it just has some features that improve package installing speed like caching previously downloaded packages to some global local storage.

by Tymur Levtsun at Thu Jan 19 2023 01:51:32 GMT+0000 (Coordinated Universal Time)

Wednesday, January 18, 2023


David Humphrey

Multipass VMs on macOS

I'm in the process of developing a demo for my students.  I need to show them how we'll do Docker based deployments in the cloud, and I wanted to quickly create a small cluster of VMs on my M1 MacBook Pro.  I debated doing it in AWS, but decided that for the purposes of this group, using local VMs would be more accessible.

So I took another look into the macOS M1/M2 virtualization ecosystem.  It's been a while since I explored this space, and the last time I looked, it wasn't really ready for Apple Silicon.

First, I looked at VMware Fusion.  I've used (and liked) VMware in the past.  It looks like this will cost something like ~$150-$200.  So I looked into Parallels, and this is similar in cost, but per year vs one-time! I'm sure that Parallels has an amazing user experience, but it's not worth it to me.

Next I went to look if my old friend VirtualBox had M1/M2 support yet. It turns out that there is a developer preview!  I downloaded and installed it, then started installing a VM.  It crashed 4 times before I gave up.

Next I tried UTM, which I've read about but never used.  It's based on QEMU, but you can also opt into using experimental support for Apple's native M1/M2 virtualization kit.  I downloaded it and created a few VMs.  Mostly, it worked great.  I found that I could use the experimental Apple virtulization vs. qemu with no problems (zero crashes).  But I couldn't get the networking between VMs and the host to ever work exactly how I wanted.

Finally, I decided to go looking for the Ubuntu virtualization tool I'd read about a while ago.  It took me a while to Google my way to success, but eventually I found Canoncial's Multipass. I'd read about this previously while researching my AWS course (I think while reading about cloud-init).

Multipass turned out to be exactly the tool I needed.  It's a little like using Docker, with a single command line tool and macOS menu bar app to control starting/stopping and shelling into VMs.

In my case, I want to spin up 3 Ubuntu VMs configured to use Docker.  I can do it like so:

$ multipass launch --name manager docker
$ multipass launch --name worker1 docker
$ multipass launch --name worker2 docker

This pulls the images and you get VMs with 1GB RAM, 5GM disk, and 1CPU (all easily configurable via args to launch), and your home directory mounted inside the VMs.  You can get shell access like so: multipass shell worker1 and when you're done, multipass stop or multipass delete to remove your instance(s).  And the networking all works without hassle!

The docs are excellent, and I ran into zero issues while building my demo.  I even filed a feature request, and was greeted immediately by the community with a positive response.

I'd recommend checking out Multipass.  It's impressive.

by David Humphrey at Wed Jan 18 2023 22:52:58 GMT+0000 (Coordinated Universal Time)


Taimoor Dawami

OSD700 - Setting up Telescope and more

Last semester in OSD600, we were introduced to the Telescope project, maintained by developers at Seneca College. It was a daunting experience to set it up locally initially, as I ran into issues with authentication and correctly seeding the Postgres database. With support from the project's maintainers, I could run Telescope properly on my operating system and contribute to both the client side and the project's backend.

Steps I take to Start Telescope

  1. Load Docker Desktop client
  2. Run the pnpm services:start script
  3. Stop parser using Docker Desktop client
  4. After all the containers are started, run: cp ./src/db/env.example .env
  5. Run pnpm db:init script
  6. Restart parser using pnpm services:start parser

Memory Issues

I have no issues running Telescope on my machine, except that specific containers consume too much memory space, which makes my system lag. I also helped one of my peers connect to the Telescope's backend locally.

Goals

This semester, I look forward to familiarizing myself more with microservices and Docker, so I am comfortable using these technologies in the future.

by Taimoor Dawami at Wed Jan 18 2023 11:56:00 GMT+0000 (Coordinated Universal Time)