Aarif Sumra

Md Abdur Rafi Ibne Mahmood

Abdur Rafi

Abinash Ghosh

Md. Abu Farhad

Adrian Macarenco

Ahmed Madkor

Ahmed Naseef Chowdhury

Akito Ando

Alexander Bondarenco

Amdadul Hoque

An Wonho

Andrei Hogea

Andrew Bailly Lloyd

Antonin Joly

Aquibur Rahman Khan

Arifena Tamrin

Md. Arman Morshed

Arman Morshed

S M Asad Rahman

Md. Aslam Hossin

Aslam Hossin

Md Atiqur Rahman

Atiqur Rahman

Azat Martirosyan

Bob De Kort

Brett R. Toomey

Carmen Mihaila

Cheryl Cheng

Chris Combs

Christian Weinberger

Christofer Henriksson

Cuong Le

Daisuke Hirata

Daisuke Oba

Patrick Dan Lacuna

Danny Lee

Deniz Tunç Tuncer

Dennis Haulund Nielsen

Dinakar Maurya

Dipanjal Maitra

Dominik Hádl

Dominik Ringler

Dongyu Wang

Eduard Ablekimov

Emelie Lindh

Enric Macias Lopez

Esmaeil Abedi

Fahim Ahmed

Faisal Ahmed

Faisal Islam

Felipe Chaves

Ferdous Al-Quraeshi

Genita Tahiri

Go Nakano

Graham Campbell

Golam Shakib Khan

Gwanmuk Kang

Hans Hofner

Harol Higuera

Hassan Saleh

Heidi Puk Hermann

HieuLuan Quan

Hiroki Saito

Hitoshi Hasumi

HongYang Chen

Hui Zhou

Inna Marchenko

Mohammed Istiaque ISLAM

Jahid Hasan Naiem

Jakob Mygind

James von Hagel

Japan Security Testing Team

Jigna Patel

Jinma Yamashita

Jiri Bucek

Joao Alves

Johnny Sørensen

Jonas Stokholm Olsen

Jonas Schwartz

Jose Nieto

Juan Eduardo Castaño Nestares

Juraj Longauer

Kamel Mohamad

Kento Morita

Khairullo Abdulaev

Kiyotaka Kunihira

Kousar Sharmin JENNY

Kun Luo

Li Huang

Lin Chen

Diarmaid Lindsay

Lucas Sales

Luis Forero

Mahamud Hasan Rashel

Mahmudul Hasan

Majid Mvulle

Marius Constantinescu

Markus Rubey

Marthy Maisog

Masumi Kanazawa

Matko Smoljan

Mayumi Ishiyama

Md. Mazharul Islam


Large Language Models are extraordinary generalists, but they fail predictably in two cases: when asked about information they have not seen during training, and when that information has changed since their cut-off date.

Fine-tuning, the practice of further training the model on your own data so it absorbs the specifics of your domain, is one answer to those two failures. It is expensive, slow to iterate, and becomes outdated the moment the underlying data changes. For most teams, what you actually want is a way to give the model fresh, private context at query time. That is what Retrieval-Augmented Generation (RAG) does.

This post walks through RAG in three layers. First, the concepts: what it is, when it wins. Then the best practices that decide whether your prototype survives contact with real users. Finally, a hands-on build on AWS with Amazon Bedrock and OpenSearch Serverless.

## What Is RAG?

The idea is simple: when a user asks a question, you first **retrieve** a small set of relevant documents from a corpus you control, **augment** the prompt with those documents as context, then **generate** the answer with the LLM grounded in that context.

The model has not learned anything new. It has been given a better question, one with the answer already in front of it.

## Anatomy of a RAG Pipeline

Every RAG system has two paths.

**Ingestion path** runs offline, when your corpus changes:

1. **Chunk** documents into passages small enough to fit comfortably in the LLM context window.
2. **Embed** each chunk into a vector using an embedding model.
3. **Index** the vectors in a store that supports approximate nearest-neighbor search.

**Query path** runs per user request:

1. **Embed** the user's question with the same embedding model.
2. **Retrieve** the top-k most similar chunks from the vector index.
3. **Generate** the answer, passing the retrieved chunks as context in the prompt.

## RAG vs Fine-Tuning

| Use case | Best fit | Reason |
|---|---|---|
| Private or domain docs Q&A | RAG | Facts are fetched at query time, so answers stay fresh and sourceable. |
| Consistent tone, style, or format | Fine-tuning | The model learns the behavior directly. |
| Fast-changing support knowledge base | RAG | Reindexing is simpler than retraining. |
| Specialized domain writing | Often both | RAG supplies facts; fine-tuning helps with phrasing and conventions. |
| Strict output schema | Fine-tuning or structured prompting | Behavior control matters more than knowledge retrieval. |

RAG and fine-tuning are not mutually exclusive; many production systems do both. Fine-tune for behavior; retrieve for knowledge.

![RAG architecture overview]({{ sitePrefix }}/assets/img/articles/2026-06-23-RAG-From-Concepts-to-a-Bedrock-and-OpenSearch-Serverless-Build/rag-architecture-overview.eng.png)

## Best Practices

Retrieval quality is the dominant lever. The practices below are the ones that matter most.

1. **Ingest cleanly with structure-aware chunking and rich metadata.** Get text extraction right before chunking: PDFs need a text-layer read with an OCR fallback for scanned pages; spreadsheets and HTML need format-specific parsers. Prefer boundary-aware splitting (page breaks first, then paragraphs, then sentences), falling back to fixed-size only when no structure is available. 500 tokens with 50-token overlap is often a reasonable starting point, but tune by document type: smaller chunks for FAQ and support content, larger for dense technical or legal text. Always keep metadata with each chunk (source URL, section, page, last-updated) so you can filter and cite.

2. **Pin the embedding model and version per index.** An embedding model maps each chunk to a fixed-size vector of floating-point numbers; the *dimension* is the vector's length (Titan v2 outputs 1024 dimensions). Embedding spaces are not interchangeable, so changing the model means reindexing the entire corpus. Tag each index with the model name, version, and dimension as metadata so query-time embeddings cannot mismatch. Respect the model's input limit: long chunks get silently truncated unless you split them first.

3. **Retrieve with hybrid search, reranking, and a tuned kNN index.** Vector search alone struggles with proper nouns, exact phrases, and rare terms. Combine vector similarity with BM25 keyword scoring for **hybrid search**. Tune top-k per query type; it is not a constant. Add a **reranker** (a cross-encoder like Cohere Rerank or a fine-tuned one of your own) over the top 20 to 50 candidates and keep the top 3 to 5 after reranking. The kNN index has parameters worth tuning: HNSW `m` (graph connectivity, default 16), `ef_construction` (build accuracy, commonly 256 to 512), and `ef_search` (query-time accuracy vs latency). Follow your embedding model's documented similarity metric (`cosinesimil` for normalized embeddings is common; some models prescribe dot product or L2).

4. **Design refusal, citation, and validation into the response path.** Decide explicitly what happens when retrieval returns nothing relevant: the model should refuse, not improvise. Number your context chunks and instruct the model to cite them: "Answer using only the context. Cite sources as [1], [2]." For strict output schemas, prefer the model's function-calling or structured-output API over prompt heuristics or fine-tuning, and combine it with a downstream validator for anything that drives programmatic actions. Sanitize untrusted retrieved text: it can carry prompt injection, so strip control tokens and filter PII out of both indexed content and outputs.

5. **Evaluate, budget, and operate.** You cannot improve what you do not measure. Build a small golden set (50 to 200 question-answer pairs) on day one, and measure two things separately: **retrieval recall@k** (is the right chunk in the top-k?) and **generation faithfulness** (does the answer rely only on retrieved context?). RAG is metered: embedding 100k chunks at $0.10/1M tokens is one cost; generating answers with a frontier model is another. Set a latency budget per stage (embed: <100ms, retrieve: <50ms, generate: <2s) and instrument each one. Decide how fresh the index needs to be: incremental upserts on write are simpler than full nightly reindexes but require more plumbing.

## Building It on AWS

Let's walk through a minimal end-to-end RAG service on AWS using Amazon Bedrock for embeddings and generation, and Amazon OpenSearch Serverless for vector retrieval.

![AWS RAG architecture]({{ sitePrefix }}/assets/img/articles/2026-06-23-RAG-From-Concepts-to-a-Bedrock-and-OpenSearch-Serverless-Build/aws-rag-architecture.eng.png)

**Components:**

- **S3**: raw documents land here.
- **Ingester Lambda**: triggered on S3 PutObject; chunks, embeds via Bedrock, indexes into OpenSearch.
- **OpenSearch Serverless**: vector collection of type `VECTORSEARCH`.
- **API Gateway + Query Lambda**: public query endpoint; embeds the question, runs kNN, calls Bedrock with the augmented prompt.
- **Bedrock**: `amazon.titan-embed-text-v2:0` for embeddings, `anthropic.claude-3-5-sonnet` for generation.

### OpenSearch Serverless (Terraform)

```hcl
resource "aws_opensearchserverless_security_policy" "encryption" {
  name = "${var.app.prefix}-rag-enc"
  type = "encryption"
  policy = jsonencode({
    Rules = [{
      Resource     = ["collection/${var.app.prefix}-rag"]
      ResourceType = "collection"
    }]
    AWSOwnedKey = true
  })
}

resource "aws_opensearchserverless_collection" "rag" {
  name = "${var.app.prefix}-rag"
  type = "VECTORSEARCH"

  depends_on = [aws_opensearchserverless_security_policy.encryption]
}
```

Network and data-access policies follow the same pattern. The collection exposes an HTTPS endpoint your Lambdas authenticate to via SigV4.

### Ingester Lambda (Go)

```go
func handler(ctx context.Context, evt events.S3Event) error {
    for _, rec := range evt.Records {
        body, err := s3Get(ctx, rec.S3.Bucket.Name, rec.S3.Object.Key)
        if err != nil {
            return fmt.Errorf("s3 get: %w", err)
        }

        chunks := chunkText(string(body), 500, 50)

        vecs, err := embedAll(ctx, chunks, 8)
        if err != nil {
            return fmt.Errorf("embed: %w", err)
        }

        if err := aossBulkIndex(ctx, rec.S3.Object.Key, chunks, vecs); err != nil {
            return fmt.Errorf("index: %w", err)
        }
    }
    return nil
}
```

`embedAll` runs `bedrock:InvokeModel` calls in parallel with a bounded worker pool and returns vectors in chunk order. `aossBulkIndex` issues one SigV4-signed `POST /_bulk` for all chunk/vector pairs. Each indexed document has shape `{"text": chunk, "embedding": vec, "source": key}`.

### Query Lambda (Go)

```go
func handler(ctx context.Context, req events.APIGatewayProxyRequest) (*events.APIGatewayProxyResponse, error) {
    var body struct {
        Question string `json:"question"`
    }
    if err := json.Unmarshal([]byte(req.Body), &body); err != nil {
        return utils.BadRequest("invalid json"), nil
    }

    qvec, err := bedrockEmbed(ctx, body.Question)
    if err != nil {
        return nil, fmt.Errorf("embed query: %w", err)
    }

    chunks, err := aossKNN(ctx, qvec, 5)
    if err != nil {
        return nil, fmt.Errorf("retrieve: %w", err)
    }

    prompt := buildPrompt(body.Question, chunks)
    answer, err := bedrockGenerate(ctx, prompt)
    if err != nil {
        return nil, fmt.Errorf("generate: %w", err)
    }

    return utils.SuccessResponse(200, map[string]any{
        "answer":  answer,
        "sources": chunkSources(chunks),
    }), nil
}
```

`buildPrompt` numbers the retrieved chunks and instructs Claude to cite them. `aossKNN` runs an OpenSearch `knn` query against the `embedding` field.

### Suggested Next Steps

- **Add reranking.** Plug a cross-encoder call between `aossKNN` and `buildPrompt`.
- **Add hybrid search.** OpenSearch supports it natively: combine a `knn` query with a `match` clause and tune the weights.
- **Add evaluation.** Keep a small set of test questions with known good answers and rerun them after each change, so you can tell whether retrieval and answers are getting better or worse.

## Conclusion

RAG is one of the highest-leverage patterns in applied LLM work: easy to start, hard to get right at scale. The concepts are simple, the best practices are where production lives, and the AWS build above is a starting line, not a finish.

If you build on this, the two follow-ups worth their own posts are **evaluation** (how to know your retrieval is actually getting better) and **reranking** (the single biggest precision lever after hybrid search). Those are the two areas that usually decide whether a RAG system stays a demo or becomes reliable in production.

## References

- [Amazon Bedrock: User Guide](https://docs.aws.amazon.com/bedrock/latest/userguide/)
- [Amazon OpenSearch Serverless: Vector Search](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-vector-search.html)
- [Anyscale: Building RAG-based LLM Applications for Production](https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1)


RAG: From Concepts to a Bedrock + OpenSearch Serverless Build


I've been building Flutter apps professionally for years now, and I've seen the backend landscape evolve from Firebase to Supabase to custom Node.js APIs. Each solution came with trade-offs: Firebase locked you in, Supabase was great until you needed custom logic, and maintaining a separate Node.js backend meant context-switching between TypeScript and Dart.

To be honest, I wasn't actively looking for a new backend solution. But Serverpod kept showing up—in my feeds, in Flutter newsletters, in conversations with other developers. The buzz was real.

So I blocked out a weekend to try it. No expectations, no pressure. Just a genuine attempt to see if the hype had any substance. Here's what I found.

---

## Setup: The First Surprise

I expected the usual setup friction: Docker configs, environment variables, database connection strings, authentication middleware. Sure, I could ask Claude or Cursor to help with boilerplate, but even with AI agents speeding things up, getting a backend stack running still means an hour of setup before writing actual business logic.

Instead:

```bash
$ serverpod create weekend_experiment
Creating project...
✓ Server setup complete (packages/weekend_experiment_server)
✓ Client library generated (packages/weekend_experiment_client)
✓ Flutter app created (weekend_experiment_flutter)
✓ Database migrations ready
✓ Docker compose configured
```

The project structure was immediately familiar to any Flutter developer:

```
weekend_experiment/
├── weekend_experiment_server/     # Backend code
├── weekend_experiment_client/     # Auto-generated API client
├── weekend_experiment_flutter/    # Flutter app
└── docker-compose.yaml            # PostgreSQL + Redis ready
```

I ran **docker compose up** in the server directory. PostgreSQL and Redis spun up. Then **dart run bin/main.dart --apply-migrations** to initialize the database.

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2026-02-03-Serverpod-for-a-Weekend-The-Good-The-Bad-and-The-Surprising/serverpod-architecture.webp>
  <figcaption>Figure 1: Serverpod's unified architecture - one codebase, full type safety</figcaption>
</figure>

---

## The Good: What Actually Impressed Me

### 1. The ORM That Doesn't Fight You

I've written enough SQL to know what good database tooling feels like. I've also suffered through ORMs that generate horrific queries and make simple things hard.

I created a model file to test Serverpod's ORM:

```yaml
# user.spy.yaml
class: User
table: user
fields:
  name: String
  email: String
  age: int?
  premium: bool
  createdAt: DateTime

indexes:
  - fields: email
    unique: true
```

Ran **serverpod generate**. The framework generated:

- A Dart class with all fields
- JSON serialization/deserialization
- Database access methods
- Client-side models
- Migration files

Here's where it got interesting. The query API felt _natural_:

```dart
// Find all premium users over 25, ordered by creation date
final users = await User.db.find(
  session,
  where: (t) => t.premium.equals(true) & t.age.greaterThan(25),
  orderBy: (t) => t.createdAt.desc(),
  limit: 20,
);

// Complex join with relations
final postsWithAuthors = await Post.db.find(
  session,
  where: (t) => t.publishedAt.isNotNull(),
  include: (t) => [t.author, t.comments],
);
```

**Type-safe. No string column names. No SQL injection possible. IntelliSense shows available fields.**

I tested it with a deliberately broken query:

```dart
where: (t) => t.nonExistentField.equals(true)  // Compile error!
```

The Dart analyzer caught it immediately. No runtime surprises. No debugging production issues caused by typos.

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2026-02-03-Serverpod-for-a-Weekend-The-Good-The-Bad-and-The-Surprising/orm-query-flow.webp>
  <figcaption>Figure 2: Type-safe queries from Dart expressions to optimized PostgreSQL</figcaption>
</figure>

### 2. Real-Time That Actually Works

Most real-time implementations involve setting up WebSocket servers, handling connections/disconnections, managing subscriptions, dealing with reconnection logic, and synchronizing state.

I wanted to test Serverpod's streaming capabilities with a simple counter.

**Server endpoint:**

```dart
class CounterEndpoint extends Endpoint {
  Stream watchCounter(Session session) async* {
    for (int i = 0; i < 100; i++) {
      await Future.delayed(Duration(seconds: 1));
      yield i;
    }
  }
}
```

**Client usage:**

```dart
client.counter.watchCounter().listen((count) {
  print('Counter: $count');
});
```

That's it. Serverpod handles WebSocket connection management, automatic reconnection on network loss, backpressure when the client is slow, and clean stream disposal.

For database-driven real-time updates:

```dart
class ChatEndpoint extends Endpoint {
  Stream watchRoom(Session session, int roomId) async* {
    await for (var message in Message.db.watch(
      session,
      where: (t) => t.roomId.equals(roomId),
    )) {
      yield message;
    }
  }
}
```

Any database change to messages in that room automatically streams to all connected clients. No manual pub/sub setup. No Redis channels. No message queues.

### 3. Code Generation That Makes Sense

Serverpod's generation is straightforward: models defined in YAML, code generated from your server structure. The output is readable Dart—no magic, no abstractions hiding complexity. When something's wrong, you can actually see what's happening.

```bash
$ serverpod generate
Analyzing server...
✓ Found 5 models
✓ Found 3 endpoints
Generating...
✓ Client library (packages/weekend_experiment_client/)
✓ Protocol models
✓ Database migrations

Time: ~5 seconds
```

### 4. Authentication Without the Tears

Authentication setup has gotten easier over the years—plenty of libraries handle OAuth flows. But integrating them still means configuration, testing callbacks, and making sure everything works across environments.

Serverpod includes **serverpod_auth** module with support for Google, Apple, and Email authentication.

**Server setup:**

```dart
import 'package:serverpod_auth/server.dart';

// Authentication is configured through the serverpod_auth package
```

**Client setup:**

```dart
import 'package:serverpod_auth/client.dart';

// Sign in with Google
final result = await client.auth.signInWithGoogle();
if (result.success) {
  print('Welcome ${result.userInfo.userName}');
}
```

The session automatically includes authenticated user:

```dart
class ProfileEndpoint extends Endpoint {
  Future getMyProfile(Session session) async {
    final userId = session.userId;
    if (userId == null) throw Exception('Not authenticated');

    return await Profile.db.findById(session, userId);
  }
}
```

No JWT parsing. No middleware chains. No authorization header management.

_(Pro tip: If you add Google Sign-In to your iOS app, don't forget Apple Sign-In too, or Apple will happily reject your app.)_

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2026-02-03-Serverpod-for-a-Weekend-The-Good-The-Bad-and-The-Surprising/authentication-flow.webp>
  <figcaption>Figure 3: Built-in authentication with Google, Apple, and Email</figcaption>
</figure>

### 5. The Development Experience

This is subtle but important: **everything is Dart**.

- Server code? Dart.
- Client code? Dart.
- Database models? Dart (via YAML, but feels like Dart).
- Tests? Dart.
- Tooling? Dart CLI.

Writing both client and server in Dart eliminates language context switching. The tooling, patterns, and conventions are consistent across the stack. You can move between frontend and backend work without the cognitive overhead of switching ecosystems.

---

## The Bad: Where It Falls Short

Here's where things weren't perfect.

### 1. Smaller Package Ecosystem

The third-party package ecosystem is limited compared to Express or Django. While you'll find some community packages (like Stripe integrations), you won't have the same breadth of options. Need a specific CMS integration? Elasticsearch connector? Twilio wrapper? You'll likely be building it yourself or adapting existing Dart packages.

AI coding assistants can help with general Dart patterns, but their Serverpod-specific knowledge is limited. They're useful for standard CRUD operations and basic server patterns, but you'll be reading documentation and source code more than you might with mainstream frameworks.

If your project heavily depends on integrating with specialized services or requires extensive third-party tooling, account for additional implementation time.

### 2. The Learning Curve for Complex Features

The basics are smooth. Creating endpoints, defining models, simple queries - all intuitive.

Advanced features? The documentation assumes you understand the underlying concepts.

Complex authorization patterns, advanced transaction handling, and intricate database relationships require piecing together solutions from Github issue and AI.

### 3. Migration Complexity

Database migrations work well for simple changes—adding nullable fields, creating tables. Complex changes require manual intervention:

- Data transformations
- Renaming columns while preserving data
- Splitting tables
- Complex constraint changes

The migration system doesn't automatically handle these scenarios. You can manually edit the generated SQL migrations, but this reduces some of the type-safe ORM benefits and requires careful testing.

### 4. Production Documentation Gaps

Getting started locally? Perfect.

Deploying to production? The docs cover Docker basics and provide a health check endpoint, but you'll need to figure out the production DevOps details yourself:

- Kubernetes-specific configurations (liveness/readiness probes, HPA)
- Graceful shutdown for zero-downtime deployments
- Database backup and disaster recovery strategies
- Log aggregation and centralized monitoring
- Production-grade security hardening

There's a community [serverpod_vps](https://pub.dev/packages/serverpod_vps) package that helps, and GitHub discussions fill some gaps, but comprehensive production operations guides would be valuable.

### 5. Performance Data

[User testimonials](https://serverpod.dev) mention Serverpod handling 100,000+ daily requests in production, and there's a community benchmark comparing it to other frameworks. But comprehensive performance documentation is sparse.

- Latency distribution under various loads
- Horizontal scaling patterns
- Memory footprint under load
- Database connection pooling best practices
- Caching strategy guidelines

This makes capacity planning for larger applications more uncertain.

---

## The Surprising: What I Didn't Expect

### 1. The File Upload System Is Actually Good

Most file upload implementations are painful. Serverpod surprised me here.

```dart
// Server endpoint
class FileEndpoint extends Endpoint {
  Future getUploadDescription(Session session, String fileName) async {
    return await session.storage.createDirectFileUploadDescription(
      storageId: 'public',
      path: fileName,
    );
  }
}

// Client uploads directly to cloud storage
final uploadDescription = await client.file.getUploadDescription('photo.jpg');
await uploader.upload(uploadDescription, fileBytes);
```

This generates presigned URLs for direct cloud storage uploads. No file data passing through your server. Supports S3 and Google Cloud Storage.

**I didn't expect this level of sophistication in a young framework.**

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2026-02-03-Serverpod-for-a-Weekend-The-Good-The-Bad-and-The-Surprising/file-upload-flow.webp>
  <figcaption>Figure 4: Efficient file uploads with presigned URLs</figcaption>
</figure>

### 2. The Caching Layer Is Built-In

Redis integration isn't bolted on—it's part of the core API:

```dart
// Cache user profile
await session.caches.redis.put(
  'profile:$userId',
  profile,
  lifetime: Duration(minutes: 5),
);

// Retrieve from cache
final cached = await session.caches.redis.get('profile:$userId');
```

Type-safe caching with automatic serialization. The cache is session-scoped, available everywhere without manual dependency injection.

### 3. Future Calls for Async Background Jobs

I needed to send an email after user registration without blocking the response:

```dart
class UserEndpoint extends Endpoint {
  Future register(Session session, User user) async {
    final created = await User.db.insertRow(session, user);

    // Fire and forget - executes in background
    await session.messages.sendFutureCall(
      'email',
      'sendWelcome',
      {'userId': created.id},
      delay: Duration(seconds: 10),
    );

    return created;  // Response sent immediately
  }
}
```

These "future calls" are persisted to PostgreSQL, survive server restarts, and provide exactly-once execution semantics. No AWS SQS, No RabbitMQ.

**This replaced what would normally require a separate message queue infrastructure.**

---

## The Verdict: Would I Use It Again?

After a weekend of exploration, here's my honest assessment:

### Use Serverpod When:

- Building Flutter-first applications where the entire team works in Dart
- Type safety across the full stack is a priority
- Real-time features are core to the product (chat, collaboration, live updates)
- Development velocity matters more than ecosystem maturity
- The team prefers staying in a single language ecosystem

### Consider Alternatives When:

- Third-party integrations are extensive and specialized
- The team spans multiple languages and stacks (Python ML services, Go microservices, etc.)
- Production deployment requires comprehensive operational documentation
- Integration with legacy systems or proprietary APIs is significant
- The project needs proven scalability patterns at massive scale

## Conclusion

Serverpod is legitimately impressive for what it is: a modern, full-stack Dart framework that eliminates significant boilerplate while maintaining type safety from client to database. The code generation is clean, the ORM is practical, and real-time features work without managing WebSocket complexity.

The ecosystem is young. Documentation has gaps. You'll write more integration code than with established frameworks. But for Flutter teams building real-time applications, the developer experience is genuinely productive once you're past the initial learning curve.

**Would I recommend it to my teams?**  
Yes, with clear expectations about ecosystem maturity and integration limitations.

**Would I choose it over Node.js + Express?**  
If the team knows Flutter? Absolutely. If they don't? Probably not.


Serverpod for a Weekend - The Good, The Bad, and The Surprising


This article is the 15th entry in the [D-Plus🐬 Development Productivity Community Advent Calendar 2025](https://qiita.com/advent-calendar/2025/d-plus).
Day 14's article is... wait, nobody's there???

I had the opportunity to speak at D-Plus in Osaka back in May with a presentation called [Task Management for Engineering Managers](https://speakerdeck.com/kuchitama/task-management-of-engineering-manager). Although I've only attended once, it was such a warm and welcoming community that I'd love to participate again when I can balance it with my household and parenting responsibilities.


## Introduction

Recently, I attended a training session for middle managers at Monstarlab.
During the training, I had the chance to reflect on my people management experience, and I had a shocking realization.

**What I thought was people management turned out to be performance management.**

In this article, I'd like to share that realization and what I learned from the training.

## Monstarlab's Four Management Domains

In the tech industry, when it comes to management, Daichi Hiroki's "[Four Ps](https://levtech.jp/media/article/column/detail_661/)" framework is well known. It organizes an engineering manager's areas of activity into four categories: People, Project, Product, and Platform (Technology).

At Monstarlab, we define manager roles across four domains—not limited to engineering:

1. **People Management** - Supporting team members' growth
2. **Performance Management** - Managing results and productivity
3. **Strategy Management** - Planning and executing strategy
4. **Philosophy Management** - Instilling organizational culture and values

(As a side note, if only we could rename "Strategy" to something starting with "P," we'd have our own Four Ps... How about Portfolio Management?)

## What I Thought Was My People Management

As an engineering manager, I believed I was genuinely engaging with my team members.

- Conducting regular 1-on-1s
- Checking in on project status
- Identifying project challenges and brainstorming solutions together
- Making sure no one feels isolated

I even gave a presentation titled [Connecting People and Organizations Through "En" - Value Creation and Unlocking Potential as an EM in Contract Development](https://speakerdeck.com/kuchitama/ren-tozu-zhi-no-en-wojie-bu-shou-tuo-kai-fa-emnojia-zhi-chuang-chu-toqian-zai-li-noyin-kichu-si), discussing team management with a focus on three "Ens": Engagement, Empowerment, and Encouragement. I thought I was doing a decent job at people management—drawing out team members' potential and alleviating their anxieties.

<iframe class="speakerdeck-iframe" frameborder="0" src="https://speakerdeck.com/player/983d4dfdffe04b15bc7bf7e72ca49dbd" title="人と組織の&quot;エン&quot;を結ぶ - 受託開発EMの価値創出と潜在力の引き出し" allowfullscreen="true" style="border: 0px; background: padding-box padding-box rgba(0, 0, 0, 0.1); margin: 0px; padding: 0px; border-radius: 6px; box-shadow: rgba(0, 0, 0, 0.2) 0px 5px 40px; width: 100%; height: auto; aspect-ratio: 560 / 315;" data-ratio="1.7777777777777777"></iframe>


But as I reflected during the training, I realized **I wasn't actually engaging with the people themselves—I was engaging with their "work."**

By focusing on project status and challenges, I was essentially concentrating on improving productivity and performance. In other words, **I was doing performance management.**


## What I Learned About True People Management

People management is about **supporting team members' growth.** To do this effectively, it's crucial to understand each member's **Will (what they want to do), Can (what they can do), and Must (what they need to do).**


<figure>
  <img src="{{ sitePrefix }}/assets/img/articles/2025-12-15-PeopleManagementTrainingNotice/will-can-must.webp" alt="Will Can Must diagram">
</figure>


During the training, a manager from another team brilliantly drew out the Will, Can, and Must from someone playing the role of a team member. This person had a strong background in branding design, and it made sense—branding is all about extracting and organizing strengths and weaknesses.
I found it fascinating that such skills translate so well to people management.

I learned that by truly engaging with team members and helping them articulate their Will, Can, and Must, we can effectively support their growth.
At the same time, I also realized that I'm not particularly good at delving into people's inner thoughts and feelings.

After the training, I found myself buying books on branding design.

I'm hoping to pick up some insights that will help me level up my people management skills—from "not good at it and not doing it" to "not good at it but can manage reasonably well."


## Summary

So that's the story of how I thought I was focusing on people management as a manager, when I was actually focusing on productivity ≈ performance.

Of course, performance management is important work too. But realizing that I was missing the perspective of supporting team members' growth was a significant takeaway.

During the training, another manager shared their experience about how even if you're not a perfect manager yourself, it's okay to leverage other managers' strengths to bring out the best in your team members.
This was incredibly encouraging, especially since I had just become aware of my own people management challenges.

Going forward, I want to focus more on each team member's Will, Can, and Must during 1-on-1s and practice true people management.

## Finally

Even though I'm still very much a work-in-progress as an EM, Monstarlab is looking for people who want to grow together with us.

In particular, we're working on launching a new team initiative like this:

[A New Form of Engineer for the AI Era - What Full-Cycle Engineers Mean at Monstarlab](https://engineering.monstar-lab.com/jp/post/2025/10/07/FullCycle-Team-is-launched/)

With the spirit of "Turn Vision into Reality," we're looking for people who want to become full-cycle engineers together. If you're interested, please apply!

[Monstarlab Careers | Full-Cycle Engineer](https://hrmos.co/pages/monstar-lab-recruit/jobs/0002501)


What I Thought Was People Management Wasn't People Management at All


When I was tasked with implementing a headless CMS to primarily serve a mobile application at scale, I found myself stepping into unfamiliar territory. As a frontend developer, backend development once felt like a daunting mix of database design, API implementation, authentication systems, and deployment infrastructure.

Payload offered an approach that leveraged existing TypeScript skills while providing necessary backend capabilities. After deploying to production and maintaining the system through several high-traffic periods, this post reflects on the practical realities of building backend systems with Payload.

## What is Payload?

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2025-11-13-Frontend-Dev-Experience-with-PayloadCMS/payloadcms-wordmark.svg>
</figure>

[Payload](https://payloadcms.com/docs/getting-started/what-is-payload) is an open-source, headless content management system (CMS) built with TypeScript and Next.js. Unlike traditional CMS platforms, such as WordPress or Drupal, a headless CMS delivers content via APIs, allowing any frontend stack to consume the content.

A major selling point of Payload is its code-first philosophy. While other headless CMS, such as Contentful and Strapi, require defining schemas through their admin panels, Payload lets you define everything in TypeScript. This approach appeals to developers who are comfortable with modern Javascript frameworks and want to extend the same development experience to their backend infrastructure.

Built on top of Next.js, Payload takes advantage of the framework's server-side rendering and API routing, while also providing its own database abstraction layer. It supports both PostgreSQL and MongoDB, offers REST and GraphQL APIs, and can be deployed on serverless platforms like AWS Lambda or on traditional servers.

For frontend developers stepping into backend work, this means you can manage content, define data structures, and expose APIs all within a familiar TypeScript environment, without needing learn a completely new backend stack.

## Configuration-as-Code

Payload differentiates itself through its management of schema definitions as code, as opposed to GUI configurations or SQL statements. In Payload, `collections` are Typescript configuration objects, with strongly-typed `fields` that are automatically translated into database table columns.

```ts
export const Articles: CollectionConfig = {
  slug: "articles",
  fields: [
    {
      name: "title",
      type: "text",
      required: true,
    },
    {
      name: "body",
      type: "richtext",
    },
    {
      name: "category",
      type: "relationship",
      relationTo: "categories",
    },
  ],
};
```

From a frontend developer’s perspective, this syntax feels familiar — more like defining component props than database schemas. Additionally, because the code lives in version control, changes can be tracked, reviewed collaboratively, and rolled back if necessary.

The collection above automatically generates several artifacts:

### Database Schema

Database tables are created based on the field configurations. The collection above will produce an `articles` table with the following columns:

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2025-11-13-Frontend-Dev-Experience-with-PayloadCMS/payloadcms-collection-erd.webp>
</figure>

### REST API Endpoints

Every collection will also receive a complete REST API without additional configurations:

```
GET    /api/articles          # Retrieve list with pagination
GET    /api/articles/:id      # Retrieve single entity
POST   /api/articles          # Create new
PATCH  /api/articles/:id      # Update existing
DELETE /api/articles/:id      # Delete entity
```

Default query parameters are also provided for easy filtering and sorting. For instance, in order to retrieve the first 10 articles containing "frontend" in their title, sorted by the creation date:

```
/api/articles?where[title][contains]=frontend&sort=createdAt&limit=10
```

### TypeScript Type Definitions

Type interfaces are also generated based on the collection configuration, providing type checking and autocomplete support in modern IDEs.

```ts
export interface Articles {
  id: number;
  title: string;
  body?: {
    root: {
      type: string;
      children: {
        type: string;
        version: number;
        [k: string]: unknown;
      }[];
      direction: ("ltr" | "rtl") | null;
      format: "left" | "start" | "center" | "right" | "end" | "justify" | "";
      indent: number;
      version: number;
    };
    [k: string]: unknown;
  } | null;
  category?: {
    relationTo: "category";
    value: number | Category;
  } | null;
  createdAt?: string | null;
  updatedAt?: string | null;
}
```

This automatic artifact generation eliminates a considerable amount of boilerplate code and maintenance, making backend development faster and more reliable.

## Reducing the Backend Learning Curve

Several Payload features make backend development more approachable for frontend developers:

### Built-in Admin Panel

Payload includes a pre-built, auto-generated admin panel that leverages Next.js Hot Module Replacement for instant feedback during development. The production-ready admin panel automatically reflects TypeScript schema changes and is highly customizable for projects with complex UI requirements.

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2025-11-13-Frontend-Dev-Experience-with-PayloadCMS/payloadcms-dashboard.webp>
  <figcaption>Payload's default admin panel dashboard</figcaption>
</figure>

<figure>
  <img src={{ sitePrefix }}/assets/img/articles/2025-11-13-Frontend-Dev-Experience-with-PayloadCMS/payloadcms-create-new-article.webp>
  <figcaption>"Create New Article" page</figcaption>
</figure>

### Internationalization and Localization

Supporting multiple languages is handled through configuration rather than complex setup. Fields can be marked as `localized`, allowing content editors to manage translations directly in the admin panel. The system automatically handles locale-specific queries and provides fallback behavior when translations are missing.

```ts
export const Articles: CollectionConfig = {
  slug: "articles",
  fields: [
    {
      name: "title",
      type: "text",
      localized: true, // Enable per-locale content
    },
  ],
};
```

API requests can specify locale preferences via query parameters, and Payload handles the underlying database queries to retrieve the appropriate content version. For instance, requesting `/api/articles?locale=ja&fallback-locale=en` returns Japanese content when available, falling back to English otherwise.

### Data Relationship Modeling

Creating relationships between data structures uses the `relationship` field type, as shown in the sample collection above. Payload handles the underlying database joins and provides a functional admin UI for selecting related content.

### Hooks

Collection-level and field-level hooks (`beforeRead`, `afterRead`, `beforeChange`, `afterChange`, etc.) mimic React lifecycle methods, allowing you to implement business logic with relative ease.

```ts
export const Articles: CollectionConfig = {
  slug: "articles",
  fields: [
    {
      name: "title",
      type: "text",
      required: true,
    },
    {
      name: "slug",
      type: "text",
    },
    //...
  ],
  hooks: {
    beforeChange: [
      ({ data }) => {
        data.slug = data.title.toLowerCase().replace(/\s+/g, "-");
      },
    ],
  },
};
```

### Custom Endpoints

Beyond the standard CRUD operations, you can define custom endpoints to handle more complex workflows.

```ts
export const Articles: CollectionConfig = {
  slug: "articles",
  fields: [
    /* ... */
  ],
  endpoints: [
    {
      path: "/:id/summary", // Define endpoint
      method: "get", // Define method
      handler: async req => {
        if (!req.user) {
          return Response.json({ error: "Unauthorized" }, { status: 401 });
        }
        const article = await req.payload.findByID({
          collection: "articles",
          id: req.routeParams.id as string,
        });

        // Execute complex operation, call external API, etc.
        const summary = await generateAISummary(article.body);

        return Response.json({ success: true, summary });
      },
    },
  ],
};
```

### Access Control

Authorization can be enforced at both collection and field levels, ensuring security is handled consistently at the data layer rather than relying on UI visibility.

```ts
export const Articles: CollectionConfig = {
  slug: "articles",
  access: {
    // Allow anyone to read "published" articles
    read: ({ req: { user } }) => {
      if (user) return true;
      return { status: { equals: "published" } };
    },
    // Only allow authenticated users to create, update, or delete articles
    create: ({ req: { user } }) => Boolean(user),
    update: ({ req: { user } }) => Boolean(user),
    delete: ({ req: { user } }) => Boolean(user),
  },
};
```

This declarative approach makes access control simpler and more maintainable than manually checking permissions in every route handler. Security is enforced consistently for every request, giving you confidence that your data layer is protected.

## Caveats and Challenges

While Payload provides a compelling developer experience, moving to production surfaces several challenges worth considering.

### Database & Schema Management

During development, the TypeScript-to-database workflow feels seamless, but in production with PostgreSQL, explicit migration management is required. Payload provides migration scripts, but you need to run `payload migrate:create` for every schema change and carefully manage the order of migrations across environments. This can add coordination overhead when multiple developers are updating schemas simultaneously.

### Serverless Deployment Complexity

Deploying Payload as a serverless function can be more complex than expected. Cold starts can impact performance, but in a live production service, they are generally not an issue with typical traffic patterns and proper memory allocation. Additionally, Payload does not natively support scheduled tasks or batch jobs in serverless environments, so features like scheduled publishing require building separate infrastructure using Lambda functions and EventBridge scheduling.

### React and Next.js Integration

Since Payload is built on Next.js with the App Router, you also inherit React Server Component challenges. Custom admin panel components must carefully navigate the server/client boundary, and hydration mismatches can occur when building custom fields that interact with browser APIs. If you've struggled with Next.js hydration errors before, expect to encounter them here as well.

## Conclusion

As a frontend developer, working with Payload felt like extending my TypeScript toolset rather than learning an entirely new backend framework. Its familiar patterns, automatic API generation, and strong schema typing made backend development more approachable than expected.

There are still challenges — managing migrations, optimizing serverless deployments, and understanding the underlying database structures take deliberate effort. However, Payload narrows the gap between frontend and backend development by letting you apply existing TypeScript experience to backend systems. Payload doesn't cover everything — advanced database optimization, distributed systems, and infrastructure management require dedicated learning. But it offers a practical entry point.

For developers comfortable with React and modern build pipelines, it proves that backend systems can be built without leaving familiar territory.

---

*Payload, the Payload design, and related marks, designs, and logos are trademarks or registered trademarks of Payload CMS, Inc. in the U.S. and other countries.*


Building Backend as a Frontend Developer: Experience with Payload CMS

Real-time data processing, part 1 - AWS ECS App Mesh and Retry Strategies


# Update Flutter apps without store review


## What is Shorebird?
Shorebird is a service that allows Flutter apps to be updated directly at runtime. Removing the need to build and submit a new app version to Apple Store Connect or Play Console for review for every app update.

Flutter already comes with hot reload/restart that allows developers to quickly test code changes during development.
Shorebird is similarly used for changes in production apps already in the hands of end-users.

## Why?
Enabling production code to be updated without needing to go through the usual store review process, which greatly reduces the time required to release a critical bug fix. With finer control over app updates it would also let us develop mobile apps with smaller release windows and flexibility similar to web site deployments.

## How?
Shorebird uses a modified version of the Flutter engine that allows updated code changes to be run at runtime using the Dart Virtual Machine. Since running code in the Virtual Machine is a lot slower then regular compiled code, Shorebird is smart enough to detect only the changed code parts to minimize its usage.

Broadly there are two main concepts.  
**Release:** 
A full app binary containing the whole Flutter app, the same as you would normally build when releasing an app. With the difference it also contains a custom Flutter engine modified to run code in a virtual machine and functionality to check for updates. The release app needs to be reviewed and distributed through app stores as usual.

**Patch:** 
The patch only contains the code changes required for a Shorebird code push update. It's created by comparing the changes to the targeted release binary.

Both are created using the Shorebird CLI tool with the `shorebird release` and `shorebird patch` commands respectively. Behind the scenes this wraps the usual flutter build command.

Simple usage flow
1. Install Shorebird CLI tools
2. Register your Flutter app with Shorebird, using the shorebird init command
3. Create a new app release with the shorebird release command
4. Distribute the shorebird release app to app stores (or applicable distribution channels)
5. Modify some dart code in the app (adding new functionality/fixing a bug)
6. Create a patch with the shorebird patch command
7. After the patch has been uploaded to Shorebird servers, on the next app start the patch will be downloaded and applied.

By default the app will check for any new patches and update the app at startup. 
The update check can also be done programmatically for more advanced use cases.

### Example demo
For this example we will be following the [Getting Started](https://docs.shorebird.dev/) guide from the Shorebird documentation. 
In short we will be adding Shorebird to the Flutter counter example app, change some code and push a patch to the app.

1. First we need to install the Shorebird CLI tools by using the handy install script. 
```shell
curl --proto '=https' --tlsv1.2 https://raw.githubusercontent.com/shorebirdtech/install/main/install.sh -sSf | bash
```
For this demo I'm using a Mac setup but there are also instructions for Windows environments in the documentation. See the Getting Started link above.


2. After the CLI tools have been installed we now have to set up the Flutter project to use Shorebird using the shorebird init command.
You will also be prompted to select a name for your app. It's only used to manage your app in the Shorebird console and won't be shown to end-users.
```shell
user@my-mac shorebird_test % shorebird init
✓ No product flavors detected. (19.2s)
? How should we refer to this app? (shorebird_test) shorebird_test

🐦 Shorebird initialized successfully!

✅ A shorebird app has been created.
✅ A "shorebird.yaml" has been created.
✅ The "pubspec.yaml" has been updated to include "shorebird.yaml" as an asset.

Reference the following commands to get started:

📦 To create a new release use: "shorebird release".
🚀 To push an update use: "shorebird patch".
👀 To preview a release use: "shorebird preview".

For more information about Shorebird, visit https://shorebird.dev
✓ Shorebird is up-to-date (1.4s)
✓ Flutter install is correct (15.4s)
✓ AndroidManifest.xml files contain INTERNET permission (1 fix applied) (83ms)
✓ Has access to storage.googleapis.com (0.2s)
✓ shorebird.yaml found in pubspec.yaml assets (9ms)
```

Checking the Shorebird console we can see our app has been added.
<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-11-07-Shorebird-Code-Push-For-Flutter-Apps/app-registered-to-shorebird-console.png>
<figcaption>The shorebird_test app has been registered in the Shorebird console</figcaption>
</figure>


3. Now we can create our first Shorebird supported release binary using the shorebird release command.
Creating an apk file for Android only just to simplify things. This would need to be run again for the iOS app.
```shell
user@my-mac shorebird_test % shorebird release android --artifact apk
✓ Downloading patch... (0.4s)
✓ Extracting patch... (0.6s)
✓ Downloading bundletool.jar... (0.7s)
✓ Extracting bundletool.jar... (1.9s)
✓ Downloading aot-tools.dill... (0.2s)
✓ Extracting aot-tools.dill... (1.5s)
✓ Fetching apps (0.4s)
✓ Building app bundle with Flutter 3.24.1 (3f7041c5e9) (75.8s)
✓ Release version: 1.0.0+1 (1.3s)
✓ Fetching releases (0.4s)

🚀 Ready to create a new release!

📱 App: shorebird_test (ab486a37-ft3d-472d-l520-ba1h7e3c3801)
📦 Release Version: 1.0.0+1
🕹️  Platform: android
🐦 Flutter Version: 3.24.1 (3f7041c5e9)

Would you like to continue? (y/N) Yes
✓ Fetching releases (0.9s)
✓ Creating release (0.7s)
✓ Updating release status (0.3s)
✓ Creating artifacts (9.0s)
✓ Updating release status (1.8s)

✅ Published Release 1.0.0+1!
Your next step is to upload the app bundle to the Play Store:
/Users/christofer_henriksson/shorebird_test/build/app/outputs/bundle/release/app-release.aab

For information on uploading to the Play Store, see:
https://support.google.com/googleplay/android-developer/answer/9859152?hl=en

To create a patch for this release, run shorebird patch --platforms=android --release-version=1.0.0+1

Note: shorebird patch --platforms=android without the --release-version option will patch the current version of the app.
```

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-11-07-Shorebird-Code-Push-For-Flutter-Apps/release-added-to-shorebird-console.png>
<figcaption>Release added to Shorebird console</figcaption>
</figure>


4. Installing and running the release shows the default counter example as expected.
<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-11-07-Shorebird-Code-Push-For-Flutter-Apps/default-flutter-counter-example-app.png>
<figcaption>The default Flutter counter example app</figcaption>
</figure>

5. Now let's make some small changes to the app.  

Changing the `colorScheme` to green.
```dart
- colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
+ colorScheme: ColorScheme.fromSeed(seedColor: Colors.green),
```

Adding a decrement button
```dart
+  void _decrementCounter() {
+    if (_counter > 0) {
+      setState(() {
+        _counter--;
+      });
+    }
+  }
```
```dart
-      floatingActionButton: FloatingActionButton(
-        onPressed: _incrementCounter,
-        tooltip: 'Increment',
-        child: const Icon(Icons.add),
-      ),

+      floatingActionButton: Row(
+        mainAxisAlignment: MainAxisAlignment.end,
+        children: [
+          FloatingActionButton(
+            onPressed: _decrementCounter,
+            tooltip: 'Decrement',
+            child: const Icon(Icons.remove),
+          ),
+          const SizedBox(width: 8),
+          FloatingActionButton(
+            onPressed: _incrementCounter,
+            tooltip: 'Increment',
+            child: const Icon(Icons.add),
+          ),
+        ],
+      ),
```

6. Next we will create the patch for the above changes with the `shorebird patch` command.
```shell
user@my-mac shorebird_test % shorebird patch android
✓ Fetching apps (0.5s)
✓ Fetching releases (0.3s)
Which release would you like to patch? 1.0.0+1
✓ Fetching aab artifact (0.3s)
✓ Downloading aab (2.2s)
✓ Building patch with Flutter 3.24.1 (3f7041c5e9) (41.2s)
✓ Verifying patch can be applied to release (28ms)
[WARN] Your app contains asset changes, which will not be included in the patch.
    Changed files:
        base/assets/flutter_assets/fonts/MaterialIcons-Regular.otf
Continue anyways? (y/N) Yes
✓ Fetching release artifacts (0.9s)
✓ Downloading release artifacts (4.0s)
✓ Creating patch artifacts (4.2s)

🚀 Ready to publish a new patch!

📱 App: shorebird_test (a1ac6a37-bc3d-459d-b820-bd1f4e3c5391)
📦 Release Version: 1.0.0+1
🕹️  Platform: android [arm32 (87.71 KB), arm64 (84.91 KB), x86_64 (89.81 KB)]
🟢 Track: Production

Would you like to continue? (y/N) Yes
✓ Creating patch (1.7s)
✓ Uploading artifacts (2.8s)
✓ Fetching channels (0.3s)
✓ Promoting patch to stable (0.3s)

✅ Published Patch 1!
```
The tool detected that we added a new Material icon that won't be included in the patch.
For our demonstration we choose to continue anyways.

7. Checking the Shorebird console we can see our first patch has been added.
<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-11-07-Shorebird-Code-Push-For-Flutter-Apps/patch-added-to-shorebird-console.png>
<figcaption>Patch has been uploaded to Shorebird servers</figcaption>
</figure>


8. Restarting the app to download and apply the patch.
After restarting the app we can confirm that the new changes have been remotely updated successfully.
<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-11-07-Shorebird-Code-Push-For-Flutter-Apps/after-patch.gif
>
<figcaption>Patched changes has been applied successfully</figcaption>
</figure>

Note that the newly added button's icon is not showing. This is as expected since the Material icon asset is missing.


## Limitations and considerations 
- Shorebird can only update Dart code. So native iOS (Objective-C/Swift) and Android (Java/Kotlin) code updates are not supported. For any native code changes like plugins we are still required to prepare a new release build as normal.

- Currently assets like fonts or images are not supported but are planned to be added. Also some care needs to be considered when changing icons. Since icon assets are tree-shaken when the app is built, you can't patch an app to use a new icon that is not already used somewhere in your app.

- For now Shorebird only supports Flutter mobile apps (iOS/Android). But Flutter desktop app support is also planned to be added later. 

- Shorebird is not a replacement for existing app stores or other side loading solutions. It is only a tool for making code updates more convenient. You still need to have the app released to your users through an app store or other distribution method.

- Since new updates need to be downloaded from the Shorebird backend, apps that are exclusively offline are not supported. In fact the shorebird init command will add the required internet permissions to the Android manifest if it's missing.

- Flutter versions cannot be updated through Shorebird. For example an app released using Flutter 3.24 cannot be patched to use another version of Flutter. Conversely, since Shorebird is using a customized version of Flutter, there might be a small delay until a new Flutter version has been prepared for Shorebird.

- Both App Store and Play Console have both technical and policy rules that limit what a remote update is allowed to contain. Shorebird is built to comply with the technical limitations, but for policy rules it's up to the developers to make sure they are not broken.
For example we are not allowed to "engage in deceptive behavior" via pushed updates. Like changing the original purpose of the app. Failing to follow the content policies can result in both your Shorebird account and developer account being terminated.
More details can be found at the Shorebird’s [Q&A](https://docs.shorebird.dev/faq/#does-shorebird-comply-with-play-store-guidelines) here.

For more information I recommend checking out the [official documentation](https://docs.shorebird.dev/) and the helpful [Discord server](https://discord.gg/shorebird).


Introducing Shorebird, code push service for Flutter apps


At this year's [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), our goal was to elevate the sports experience to the next level with cutting-edge AI and machine learning technologies. With that in mind, we designed a unique solution for football fans that will open up new dimensions for watching games. We created interactive simulations for live matches that fans will be able to interact with and peer into more details behind the crucial moments in the games.

## Problems We Saw
Football fans often engage in discussions on “**what-if**” scenarios. They want to explore different scenarios in their heads, talk about them on social media, and even compete with each other. Apart from that, they cherish the memory of their favorite team winning the trophy, feel heartbroken if their team didn’t make it, and ponder what went wrong. These highly emotional experiences can be enhanced in a way that introduces healthy challenges and keeps fans engaged for years to come.

We feel that, as fans, we would like to have a more immersive experience while watching the game and engage in meaningful conversations with our friends. These open up a new social dimension to the sports we enjoy. Being able to explore the “**what-if**” scenario in a novel way is always enjoyable and worth paying for. We believe that it will be particularly interesting to see these interactions between fans, which will drive engagement to the roof.

## Solutions We Found
We are introducing UCL Max, a companion app concept for fans where not only will they be able to watch the game but also explore the “**what-if**” scenarios. Fans can sign up for a subscription to watch the game, get instant highlights and advanced match facts, interact with AI-powered simulations in **AltPlay**, and compete in the global leaderboard. Let’s dive into more details to see how this works.

## AltPlay
We use cutting-edge machine learning models to predict when a crucial event happens within a live game. As soon as we get that game footage from the cameras, we can determine the relative positions of the players and various metrics from those moments. All of these data are fed into a pre-trained machine learning model, which can be used to predict alternative behaviors of any player in any moment and position.

We have identified the following parameters to train our models:

- **Timestamp:** represents a single frame of the video.
- **Object Type:** distinguishes between players and the ball.
- **Object ID:** unique identifier for each player.
- **X-Coordinate, Y-Coordinate:** player or ball position on the field relative to the field.
- **Speed:** the instantaneous speed of the object.
- **Heading:** direction of movement in degrees.
- **Ball Possession:** Indicates the player ID with the ball (if any).
- **Pass Origin:** The player ID of the passer.
- **Pass Target:** The player ID of the intended receiver.
- **Pass Type:** Basic classification of the pass.
- **Action:** Placeholder for basic actions like shots, tackles, etc.
- **Action Type:** A new column to categorize the actions more specifically: Shot, Tackle, Dribble, Foul
- **Shot Type:** Ground, volley, header, chip, free kick, etc.
- **Shot Direction:** Angle relative to the goal or general direction (left, right, center)
- **Shot Outcome:** Goal, saved, missed, blocked
- **Tackle Location:** General pitch area (right side, midfield, penalty area, etc.)
- **Tackle Outcome:** Successful (won possession), unsuccessful, foul

We use AWS Lambda, Fargate, and DynamoDB to process the live match events as well as the previous match events. Data aggregations from these services are sent into the Amazon Sagemaker pipeline to generate inferences and train our existing models. We use AWS Lambda to pre-process the video data and perform calculations on the inferred data. AWS Rekognition is used to determine various events in the game, such as passes, tackles, goals, etc.

Here’s a diagram of the AWS stack:

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-27-Introducing-ucl-max-altplay-a-turn-by-turn-real-time-football-simulation/diagram.webp>
</figure>

Using this data, we can predict with confidence when a player makes a successful pass or releases the ball when under pressure. So we incorporate all of this information into a simulation that can be played with each turn based on the actual game. Here’s a formula to determine the pressed player:

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-27-Introducing-ucl-max-altplay-a-turn-by-turn-real-time-football-simulation/calculation.webp>
</figure>

## How It Works
Users of the UCL Max app can watch the live broadcast of the games and access the moments of the match that can be simulated. When they initiate a simulation, they are presented with a layout of the initial conditions of that moment. The real match path is already highlighted, and the users have the option to play an alternate game. Tapping on the player's position will reveal a possible path that the player can take, and based on the target player’s statistics, we simulate what’s going to happen.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-27-Introducing-ucl-max-altplay-a-turn-by-turn-real-time-football-simulation/simulation.webp>
</figure>

After every turn, the positions of the players will be updated based on the ML inference, and the user will be able to run the next step of the simulation. Once completed, the user will be able to finalize their simulation and play it back. Based on events, for example, a goal scored, if the user can simulate a goal event on an alternate path, then they will be awarded points. This simulation can be shared on social media outlets.

The more points an individual collects, the higher up they go on the leaderboard. We can decide the winner of the leaderboard after a season and offer prizes in collaboration with the sponsors.

## Technologies We Used
Here’s a list of the technologies we used to create our simulation pipeline:

- **AWS Lambda -** Data preprocessing, mathematical calculations for events such as player pressure, goal probability, sprint speed, duration, etc.
- **AWS Fargate -** Serverless cluster computing.
- **AWS API Gateway -** Provide access points to ingest data and inference output.
- **AWS DocumentDB -** High-performance JSON database to store parameter data.
- **AWS DynamoDB -** Database to store various data
- **AWS S3 -** Model archival, JSON data archival, caching.
- **Amazon SageMaker -** AI and ML pipeline.
- **AWS Rekognition -** Computer vision technologies, video analysis, and image recognition.
- **AWS Amplify -** Web and mobile app solution, access to other AWS services and streaming.
- **AWS SQS -** Messaging service between other AWS services.

## Why It’s Unique
Our AI-powered AltPlay simulation explores the alternate reality fans often crave. Also, our solution uniquely enhances the user experience and long-term engagement with sports. We believe this novel technology, social aspects, and user retention make our solution innovative.

The main target of our solution is the fans who enjoy sports and are very passionate. Also, the amount and scope of the data we designed to process can be repurposed to enhance team management and player training improvement as well.

We found that the services we are offering to our fans are unique, as we provide a way to watch the game and interact with the highlights in a completely different way. With the emergence of AR/VR platforms, we think that our solution is particularly suited to enhance the experience of watching games and discussing them with friends.

## How It Impacts Us
Users will be able to satisfy their curiosity about how they feel about the game and participate in healthy challenges among friends. This will significantly improve the sports experience and add a new dimension to social aspects. Users will get to learn more about their favorite teams and how the players work together. We believe that this will be beneficial for the fans of the sport.

## Scalability and Business Opportunities
We use a myriad of AWS services to ensure that our product remains robust and scalable from a technical perspective. Since the football fanbase is one of the largest in the world and spans across continents, we have a potential userbase that is global.
Official live broadcasts of football matches are already paid for. By subscribing to UCL Max, users will be able to enjoy the live broadcast from anywhere. We believe that fans will pay the right amount of money for live broadcasts and additional perks and features that open up a new dimension of sports. Leaderboard winners of previous seasons may be rewarded with match tickets for the next season and may receive merchandise promoting the sponsors. This also creates scope for targeted advertising for users. A monthly subscription or one-time season pass can be possible payment strategies.

Our platform and ideas are not limited to football. We can implement similar strategies to other sports such as American Football, basketball, etc. Those leagues have strong fanbases; if they are persuaded to realize the value of our simulations, we could even host betting tournaments where it’s legal. The possibilities are endless.

## Challenges We Faced
Our primary challenge was to figure out how we could satisfy the questions of what-ifs for our fans and, in turn, make that competitive. Also, we needed something economically viable. When deciding on how our simulation will work, we found that if we allow the user to simulate a whole match, this is going to be a completely different thing, like a video game. Then we thought instead of simulating the whole game, we would just play the highlights and let them simulate the specific scenarios. It felt like the right direction since we don’t talk about all the moments in a game but only some interesting moments. The next step was to determine what our simulation would look like inside the app. After a lot of brainstorming sessions, we found our inspiration and designed it.

## Final Words
MonstarHacks 2024 presented some real challenges this year. We were amazed to learn about the progress in sports that AWS had unlocked. We were so influenced by the BundesLiga Match Facts that we designed our solution based on those ideas, and AWS provided the perfect tools to aid in our solution. Overall, it was really fun and engaging, and we learned a lot from our experience and team efforts. We are looking forward to the next MonstarHacks. Until then, best wishes to all.

_Article Photo by [Image by master1305 on Freepik](https://www.freepik.com/free-photo/victory-excited-happy-family-watching-sport-match-championship-couch-home_17248535.htm#fromView=search&page=1&position=6&uuid=aff3be6d-eba9-489c-a9f6-7559c4cc029a)_


Introducing UCL Max AltPlay, a turn-by-turn real-time Football simulation


As part of Monstarlab's 4th edition of the internal hackathon, [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), the theme was to leverage cutting-edge technologies to innovate within the sports industry by developing solutions that enhance fan engagement, optimise athletic performance, and transform sports venues into smart environments.

Together with Amazon Web Services, the world’s most comprehensive and broadly adopted cloud, this event allowed us to explore over 200 fully featured services from data centers globally including AWS’s Generative AI and Machine Learning technologies.

## The Team

![Team Photos]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/TheTeam.webp

As a team of 2 architects, Ross and Graham were like a band with 2 bassists.
They were cool, but they needed some drums.

So they drafted in long-time collaborator and drinking buddy, Ian Gifford from Atom Bank (UK FinTech), who took their vision and ran with it in Figma for mockups and ident.

_Drum & Bass._

### The Problem

We decided to focus on **Performance Analytics (Athlete View)**, looking at holistic and historic sports data to try and consider insights that could improve athletic quality and wellbeing.

- Develop solutions centred around athlete data including tracking for performance, health, and safety.
- Employ machine learning to predict in-game strategies and prevent injuries.
- Generate real-time insights that can influence game-day decisions and strategies.
- Enhance scouting and recruitment processes.
- Predictive Analytics and Insights

As we’ll see, this proved more difficult in reality… The business case proposed turns this hurdle into an opportunity.

### Staying close to home

All members of Team 5 grew up within a few miles of Newcastle United Football Club and are all lifelong fans.

![NUFC Crest]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/NUFC.webp

For us, Newcastle as a place can’t be easily separated from Newcastle the club. It’s in our DNA, so choosing football was easy for us.

The 2023/24 season for Newcastle has been a nightmare due to the amount of injuries suffered by players.

This injury problem can probably be directly related to overachieving in the 2022/2023 season and having a
European competition to compete in with a limited squad. Our success has been our demise.

![Data table from BBC Sport showing NUFC incurring the most injuries and days lost this season]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/NUFCInjuries.webp

We thought - is there a way to utilise publicly available data along with historical injury data to enable a way to help predict the likelihood of an injury reoccurring?

Is there a correlation between different sources of data available?

- Fixtures & Results
- Travel Distances
- Weather
- Injury history

We decided to explore this first.

### Public Data

Data on fixtures, location, players and injuries is available along with other sources such as Weather which could be ingested - however this data is difficult to obtain in bulk.

In addition, it turns out that injury data publicly available is under-reported compared to underlying data - “The overall injury cumulative incidence of 0.63 injuries per player-season was substantially lower than in reports using other data collecting methods (ranging from 1.5 to 2.5 injuries per player-season)”.

![Publicly available injury history from transfermarkt website]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/InjuryHistory.webp

Predictive modelling of football injuries is possible given sufficient data, however the availability of the data is limited given most clubs who invest heavily in Sports Science have commercial reasons to keep that data private.

Manchester City Academy’s Head of Sports Science, for example, oversees the data collection of nine teams and over 200 players, from Under-9 through to the Under-23s.

## Data Ownership

So the question becomes - who owns the data? UK Players’ performance data could be considered to be personal data, which requires players’ explicit consent for clubs and leagues to use. The argument is that this consent may not yet been given, and that ownership by clubs and leagues has just been assumed.

![Plot of Goals per Game vs Shot Accuracy in Europe's Top 5 Leagues]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/ShotAccuracy.webp

It’s not yet something clubs or leagues have universally tried to claim ownership of, with a view to commercialising, and this is something which is currently being discussed, however in the US it’s different, as both the national American Football (NFL) and Basketball (NBA) leagues have a level of ownership over this data, more so in the NFL.

## Money is shared, but not data or insights

While top clubs have the finances to invest heavily into Sports Science, this investment is only available down to tier 4 or 5 of the EFL Pyramid and although TV Revenue does flow down, although only so far, the insights available from Sports Science does not.

![Distribution of TV Revenue by tier in the English Football LEague]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/TVRevenue.webp

This seems like a waste of potential, and likely hampers efforts to grow better footballers at lower levels in the English football community.

## Non-commercial benefits of sharing data

Given the concerns mentioned around data ownership and commercialisation - how else could the sharing of data benefit the members of the Football Association themselves directly or indirectly?

The French National Agency for Sport, together with the National Institute for Sport, Experience and Performance (INSEP) and the General Directorate for Sport, have developed the Sport Data Hub - FFS.

![French Football Federation Logo]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/FFF.webp

The project was born in 2020 with the idea of boosting the individual and collective performance of French sport in the run-up to the Olympic Games in Paris 2024. It consists of the creation of a collaborative tool for all those involved in the sports movement, (federations, athletes, coaches, technical teams, institutions and researchers), to share data that allows for aggregate comparative analysis at national and international level.

![French Olympic Committee Logo]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Olympic.webp

Who would benefit from the sharing of data, or at least these insights from data, across the entire English Football League and _who would want this_?

## Who wants this?

Below are Mission statements from both the FA (Football Association) and the PFA (Professional Footballer Association). The FA governs the UK National Game with the PFA providing Union and wellbeing support for professional players in the UK.

> Promoting health and wellbeing, and the power of teamwork, with our new strategy for 2020- 2024, we have a plan for all. Positively impacting every community across the country. Everyone can win if we build on the progress made over the previous four seasons. We’ve moved forward in every area, modernising our organisation to serve a game for all. We have a great platform to build on. We will keep pushing forward. It’s time to deliver real change.

_The Football Association (FA)_

> We believe in the unifying power of football in society, and are committed to empowering footballers to recognise their value as people, not just players. We protect players' rights, represent their views and provide support through a wide variety of educational, financial and wellbeing services.

_The Professional Footballer Association (PFA)_

## FA Vision

> From team managers and club secretaries through to players and parents, Whole Game is for the whole of grassroots football.
> <br /><br />
> Whatever your role in the game and whatever you do, Whole Game can help you to do it faster and more effectively.
> <br /><br />
> You’ll find club data – such as player contacts, qualifications, discipline records and so much more all in one place, which is great news for managers and club secretaries when it comes to club admin and monitoring player availability.
> <br /><br />
> You only have to input a player’s details once – and the system fully integrates with both Full-Time and the Matchday app to make access and management of information simple.

![The FA Vision]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Vision.webp

## The problem revealed

At this point, we realised the problem to be solved wasn’t looking at the correlations between different data sets. This could come later.

The actual problem was how to coordinate an ecosystem of different stakeholders, with different levels of access to data, to ensure that insights from the most privileged could benefit the least privileged.

> How could Professional UK Football and Grass Roots football share insights to benefit the national game in a long-term and sustainable way.

![Pivot]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Pivot.webp

Pivoting Early, but read on for our exploration of this problem and a proposition that supports our solution.

## The Solution

![AgoraMetrics Logo]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Logo.webp

_Measure, Learn, Win. Together_

> At AgoraMetrics<sup>\*</sup>, our mission is to democratise the use of advanced data analytics and machine learning in football, making cutting-edge sports science accessible to clubs of all sizes and resources all the way down to grass roots level.
> <br /><br />
> We're committed to levelling the playing field by providing smaller clubs with the insights and tools previously available only to top-tier teams.
> <br /><br />
> By joining our ecosystem of data sharing, clubs can enhance player performance, prevent injuries, and optimise team and individual management strategies, all while supporting the growth and competitiveness of football at every level.

_<sub>\* Inspired by the Greek "agora", the central public space in ancient Greek city-states, symbolising the gathering and sharing of ideas and now, sports data analytics.</sub>_

## The AgoraMetrics Proposition

![The AgoraMetrics proposal]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Proposition.webp

The AgoraMetrics proposition is aimed initially at the UK Football Association, the body that holds accountability for governance of both Professional and Grass Roots football in England.

The benefits outlined in this section detail granular positive outcomes for English football at all levels, but at a high level our mission is to Win an International competition by 2030.

We would expect **returns on investment** to include:

- Positive outcomes at Grass Roots level between years 1-3
- Positive Professional Football outcomes from years 2-5
- Positive England Team outcomes within 4-7 years
- A self-sustaining AgoraMetrics Service within years 1-4

![World Cup]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/WorldCup.webp

These outcomes assume 4 considerations to **achieve our mission**:-

- AgoraMetrics is funded by the FA from 2024-2030
- AgoraMetrics is marketed to and adopted by the Grass Roots football community extensively to drive adoption
- AgoraMetrics quantitatively improves player quality at every level of the UK game
- AgoraMetrics grass roots players take their career to professional level enabling progression ( and outcomes measurement ).

![AgoraMetrics Stakeholders]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Stakeholders.webp

## Solution Design

### Solution Landscape

![Solution Landscape]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/SolutionLandscape.webp

While data would initially flow from professional clubs down to grass roots level, data would be fed back into the system to benefit teams and players at all levels.

### Solution Detail

![Solution Detail]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/SolutionDetail.webp

The solution itself would comprise several Data Science models produced for the benefit of the community along with an app to onboard coaches, players and data which would feed back into the data layer.

### Data Mesh and ML Factory

![Data Mesh and ML Factory]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/DataMesh.webp

Several Amazon Web Services would be utilised to provide the back end Data mesh and Machine Learning models which would be exposed via secure APIs for use by the client application.

### Serverless Architecture

![Serverless Architecture]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Serverless.webp

A Serverless Architecture would be utilised to provide the Web Application layer to prioritise performance and cost effectiveness and allow scalability.

## The Agorametrics Application

### Design Influence

We were interested in using a colour scheme that resonated with players, coaches and anyone with a wider interest in football.

With this design principle in mind, we couldn’t think of a better way to start than to picture St. James’ Park (Newcastle United’s Stadium) on a Sunny Day.

![Colour Scheme]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/ColourScheme.webp

This image was used to derive a palette that could be included in a design system and includes tones for components such as alerts (orange / yellow from stewards jackets) as well as primary ident colours taken from the pitch and the sky.

Our logo is intended to convey the notion of team sport in general, with data points composing the “ball”. This ensures we’re not locked into “Football” as a sport.

### User Features

We included a finite set of App Features to discuss for the prototype. These allowed us to explore both the benefits of possible insights the Machine Learning and Data science might hunt for, and also how players and coaches might use these features.

![User Features]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Features.webp

### User Interface Design

![Green user interface]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/UserInterfaceGreen.webp

![Blue user interface]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/UserInterfaceBlue.webp

## Financials & opportunity

![Financial forecast]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Financials.webp

The AgoraMetrics product aims to provide as wide a range of returns on investment as possible.

In this model, “Data Value” could be compared with the Social Media model of “You are the product” from an end user perspective.

In AgoraMetrics case however, rather than monetising this data for corporate profit, we aim to drive financial gains back into Grass Roots initiatives ensuring Social Return on Investment which will contribute directly to the FA’s Mission.

The decision to pitch this to the FA has a leverage value, in so far as the FA governs Pro Clubs in the UK, and controls the throughput of revenue streams such as TV Rights. We would therefore expect that the FA organisation has the necessary ability to promote and project this initiative.

![]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/Opportunity.webp)

## Future Opportunities

![]({{ sitePrefix }}/assets/img/articles/2024-05-23-Agorametrics-Measure-Learn-Win-Together/FutureOpportunities.webp)

## Conclusion - The Hackathon Experience

While we both entered the weekend hoping to create a tangible working prototype utilising some cool functionality from Amazon Web Services - the time spent trying to access and retrieve data for our initial idea meant we had to pivot to a more conceptual approach.

We realised that the main problem with our idea was not the data itself - this exists and is studied in depth, but only by those who have the funds and resources to do so. Top clubs have entire teams dedicated to Data Science and utilising the latest and greatest in sports analytics - but, for good reasons, this data is held onto and not shared. However, the insights and models which can be obtained from this data can benefit lesser teams - and in return, more data can be utilised by the system to benefit all.

It is this democratisation of data, insights and models which we explored and feel fits well into the visions of both the FA and PFA and could potentially be implemented across many different sports communities.

We both look forward to the next Hack!

_Article Header from Ross Davidson, Article Photos from [NUFC](https://www.nufc.co.uk/), [transfermarkt](https://www.transfermarkt.co.uk/), [French Olympic Committee](https://olympics.com/ioc/france), [French Football Federation](https://uk.fff.fr/)_


Agorametrics: Data democratisation from Sports Science for the benefit of all 


Imagine you are a parent and want to ensure your child has the right set of guidance to explore different sports and choose any to specialize in. You dream of seeing your child compete in the Olympics in 15 years! The question is where to start!

In today’s day and age, there is hardly any platform where parents can get guidance to ensure their children are equipped to pick up any sports and get better at it. Be it recreational or professional, we don’t see this often. Moreover, we have seen athletes drop out of their game because of a lack of training and guidance. The cost involved and the difficulty in finding the best trainers in their localities further complicate the situation.

We, team Ready Player Three, wanted to address this problem in this year's [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks). Let us tell you readers how to approach the hackathon problem and how we tried to solve the issue.

![AI Generated Image]({{ sitePrefix }}/assets/img/articles/2024-05-23-AthletePro-Empowering-the-next-generation-of-champions/abstract-intro.webp

MonstarHacks partnered with Amazon Web Services (AWS), to provide our teams with challenges in 3 solution areas: Fan360, Performance Analytics, and Smart Venues. Our team made out of Saad Bin Amjad, Tanveer Hassan, and Faisal Islam, choose the Performance Analytics ("Athlete View") category aiming to help athletes and their families navigate their sports journey more effectively. Leveraging AWS technology, especially Generative AI, we aimed to build a solution that would make a tangible difference in the sports world.

We named our application AthletePro, which plans to revolutionize the sports industry with an AI-driven platform that provides hyper-personalized roadmaps for athletes and parents from recreational to elite levels, empowering the next inclusive generation of champions.

## Solution Overview

{{ youtube-link="Ahz_VbisdZk" }}

AthletePro offers a comprehensive solution to empower athletes and their families. Even before signing up for the platform, parents are given a tour of the app features and what it can offer them and their children in terms of sports. This includes sneak peeks at coaches, trainers, events, and the financial aspects involved in getting the athletes up to speed. There are curated FAQs that enable parents to make the right call in terms of exploration or being onboarded to the application.

Then we ensure that parents can sign up easily. We first focused on **Personalized Onboarding**. This feature enables parents to create profiles for themselves and their children by answering a series of questions. The questions range from their location, age, sports they want to explore, end goals if any, and budget. This helps the application tailor recommendations accordingly, as this information becomes part of the knowledge base going forward.

Parents of special needs children can provide additional information to receive more personalized recommendations.

Similar to other ed-tech platforms but specifically for sports, we showcase tailored lessons and suggest sporting activities. To support this feature, we introduced the onboarding of coaches, trainers, and organizers into our CMS-styled platform powered by Strapi. We ensure it is easily accessible, allowing users to enter the system with their certifications and facilities, which will also become part of the knowledge base going forward.

Businesses in the sports industry, such as gyms, training facilities, and equipment suppliers, can easily connect with athletes and parents through AthletePro. The bigger plan for this feature is that when governments, sports organizations, and universities are onboarded, they can sponsor athletes and their families so they can pursue their sporting journey.

Additionally, we understand that when AI generates any roadmap for vulnerable groups in society, such as children, there should be accountability and transparency. It is crucial that we provide accurate roadmaps. Therefore, we have proposed and showcased a notification system for connected coaches and trainers to first approve the content before presenting the roadmap to parents or children.

We also recognize that not everything can be taught or practiced indoors. The courses are designed so that AthletePro provides exclusive content and perks unlocked at specific stadiums or training facilities, enhancing engagement. This means the application can detect your location and then unlock the corresponding training guidance or instructions.

We understand that not everything can be taught or practiced indoors. So the courses are designed so that AthletePro provides exclusive content and perks unlocked at specific stadiums or training facilities, enhancing engagement. This means the application can detect your location and then unlock the corresponding training guidance or instructions.

![Mock Design]({{ sitePrefix }}/assets/img/articles/2024-05-23-AthletePro-Empowering-the-next-generation-of-champions/athele-pro-trainer.webp

## Target Users and Revenue Model

AthletePro caters to a diverse range of users across the sports ecosystem, including athletes, parents/guardians, coaches/trainers, government sports agencies, universities and colleges, and local businesses.

![Revenue Generation]({{ sitePrefix }}/assets/img/articles/2024-05-23-AthletePro-Empowering-the-next-generation-of-champions/business.webp

To address the needs of these diverse stakeholders, we have developed a solid revenue model for the app based on subscription-based revenue, partnerships, and sponsorships. Additionally, we engage in ethical data monetization through partnerships with research institutions, sports analytics firms, and market research companies. We believe AthletePro has the potential to expand into new markets domestically and internationally by adapting its platform to cater to different sports, cultures, and regions.

## Engineering Approach: Seamlessly Integrating the RAG Framework for Contextual Guidance

We emphasized heavily how easy this app should be for parents and athletes alike. We understand that with any GenAI-backed application, most of the time, we resort to chatbots. However, chatbots won't work here, as not everyone is suited to ask the right questions at the right time. Instead, we've innovatively integrated the RAG framework into the app's architecture, embedding contextual guidance seamlessly within the app journey itself. This user-centric approach not only enhances the overall user experience but also reinforces AthletePro's commitment to accessibility and inclusivity in sports technology.

Leveraging the retrieved data from the knowledge bank, AthletePro dynamically generates system messages that resonate with the user's current context and stage in their athletic journey. These messages are strategically woven into the app journey, serving as subtle cues that guide users towards relevant actions and decisions without overwhelming them with explicit prompts or queries.

The final piece of the puzzle involves generating prompts and recommendations based on the contextual cues provided by the system messages. AthletePro's backend formulates queries for our language model systems (LLMs), such as Anthropic's Claude in Amazon Bedrock, drawing upon the user's profile, preferences, and journey stage. These queries are designed to extract actionable insights and deliver tailored guidance that aligns with the user's unique circumstances.

![AI Flow]({{ sitePrefix }}/assets/img/articles/2024-05-23-AthletePro-Empowering-the-next-generation-of-champions/ai-flow.webp

### Core Technologies

- Strapi CMS
- Amazon Bedrock
- Amazon EC2
- Amazon S3
- Amazon RDS
- Docker
- Github
- Prototyping Tools

### RAG Framework Utilization

![User Flow]({{ sitePrefix }}/assets/img/articles/2024-05-23-AthletePro-Empowering-the-next-generation-of-champions/user-flow.webp

In the user flow diagram, the process begins with Sarah and her mother signing up for AthletePro. Sarah shares her interest in volleyball along with her preferences and any constraints. AthletePro’s system then uses this information to create a personalized roadmap recommendation. This recommendation is sent to local trainers, who review and adjust it as needed before returning it to Sarah’s mother, completing the feedback loop. This user flow demonstrates how AthletePro seamlessly integrates user data, AI-generated recommendations, and real-time communication through its backend architecture.

## Challenges Faced

One significant challenge we encountered during the hackathon was defining the scope of our idea. AthletePro had the potential to benefit various stakeholders, including athletes, parents, coaches, government agencies, and businesses. We wanted to showcase the platform's capabilities while ensuring clarity and focus. Ultimately, we decided to center our solution around the role of parents, recognizing their pivotal role in supporting young athletes.

To overcome this challenge, we proactively engaged with parents, conducting interviews to gain insights into their perspectives, motivations, and pain points regarding their children's athletic pursuits. By listening to their experiences, we were able to refine our solution and tailor it to address the specific needs of parents navigating their children's sports journey.

Additionally, we faced difficulties in accessing Anthropic's Claude2 model, a crucial subscription service, due to a waiting period and regional restrictions. To mitigate this issue, we improvised by using Amazon's Titan instead of our own knowledge base of parents and athletes. AWS Bedrock only supports Anthropic's models for using knowledge bases for now. Although we set it up, we could not test it due to this limitation.

## Conclusion

AthletePro represents a groundbreaking innovation in the sports industry, offering personalized guidance, resources, and support to athletes and parents of all backgrounds and abilities. By leveraging technology and a commitment to inclusivity, AthletePro aims to empower athletes to reach their full potential and foster a more vibrant sports community. As we continue to iterate and expand our platform, we look forward to revolutionizing the sports industry and creating a brighter future for athletes everywhere.

_Article Photo by [Seth Kane](https://unsplash.com/photos/man-riding-green-racing-wheelchair-oyZalm8fvS4)_


AthletePro - Empowering the next generation of champions; parents first approach!


The excitement of a game highly depends on the commentary that describes the in-game events. However, for individuals with sensory disabilities, especially visual or hearing impairments, enjoying sports events can be challenging. Our solution was developed as part of the 4th Edition of [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks) which is an internal hackathon hosted by Monstarlab where one of the goals was to engage fans in sports. We developed a solution to make sports enjoyable for all without any limits, especially for sensory-disabled individuals. By bridging this sensory gap, our solution — _SportsBuddy_ is proposing a new way on how to experience sports, making it both accessible and enjoyable at the same time for everyone, regardless of their abilities.

## Problem Statement

Sports fans with visual and hearing impairments face challenges in enjoying game events in detail. Visually impaired fans face difficulties in understanding in-game play, and on the other hand, hearing-impaired fans don’t get thrilled by the live commentary. These difficulties not only take out the excitement but also makes them feel that they are not part of the game.

## The Solution

_SportsBuddy_ is a revolutionary platform that opens a new door for sports enjoyment for individuals with sensory disabilities, especially with visual or hearing impairments. _SportsBuddy_ offers detailed commentary for blind individuals and sign language features for deaf individuals, ensuring an immersive sports experience for all.

![*SportsBuddy Mobile App* Flow Chart]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-main-promo.webp

<figcaption>Figure: SportsBuddy Mobile App</figcaption>

From our research, we found the challenges faced by individuals with sensory disabilities when accessing sports commentary. This involved researching the potential users, and their preferences, and identifying key pain points.

![*SportsBuddy* Mobile App Screens]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-all-screens.webp

<figcaption>Figure: SportsBuddy Mobile App Screens</figcaption>

By using the power of AWS, we used a range of cutting-edge technologies to build our solution.

## _SportsBuddy_ Flow Chart

![*SportsBuddy* Flow Chart]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-diagram-flow.webp

<figcaption>Figure: Flow Chart of the Solution</figcaption>

## How does _SportsBuddy_ help visually impaired fans?

_SportsBuddy_ provides a detailed narration of sports events, allowing visually impaired fans to follow the in-game action on the playground through audio commentary which includes descriptions of player movements, game strategies, and key moments, and even any fun moments happening on the fields.

It also offers audio descriptions of visual feeds such as player shirts, team formation, etc., helping visually impaired fans visualize the game in their minds and stay fully connected to the game.

### Smart Switch

_SportsBuddy_ uses descriptive onscreen statistics, including leaderboards, to surpass the limited coverage offered by the regular standard commentary system. Users can disable this feature if they wish to follow the match commentary only. However, enabling this feature ensures that the audience remains informed of all the key moments during the match. If an exciting event occurs, such as a goal, foul, or a red card, SportsBuddy’s smart switch feature seamlessly transitions to that event and resumes the statistics analysis afterward.

![Architectural diagram for generating detailed commentary]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-architectural-diagram-blind-tailored-commentary.webp

<figcaption>Figure: Architectural diagram for generating detailed commentary[1][2]</figcaption>

## How does _SportsBuddy_ help auditory-impaired fans?

_SportsBuddy_ offers AI-based sign language commentary for hearing-impaired fans. Those fans will follow the game action through visual interpretation, which includes sign language interpretation of game commentary, match analysis, and key moments, to ensure an immersive sports experience.

Apart from the sign language commentary, _SportsBuddy_ offers text-based commentary for those who prefer to follow the game through written match events with the descriptive text of game events, player performances, match analysis, etc.

It will show notifications on any important match events during the match time, for example, goals, fouls, cards, penalties, halftime, etc. It ensures that a hearing-impaired fan stays informed of all the key moments of a game.

The sign language avatar\[3\] is movable, so users can drag the avatar to their preferred place for optimal viewing without missing any in-game key moments. Also, users have the ability to disable the sign language avatar if they are not interested. We are planning on localizing the sign language feature to target a broader user base.

![Architectural diagram for generating sign language commentary]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-architectural-diagram-deaf-sign-language.webp

<figcaption>Figure: Architectural diagram for generating sign language commentary[4]</figcaption>

## Technology Used

To develop our solution we have used several technologies including Amazon Services.
We’ve listed them below.

**Flutter**: Flutter is an open-source framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase. We’ve developed our application _SportsBuddy_ using the Flutter framework for Android and iOS mobile devices.

**Amazon S3**: Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. We’ve used Amazon S3 for storing user data.

**DynamoDB**: Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. We’ve used DynamoDB for storing the session memory, statistics feed, sign language avatar, etc.

**AWS Lambda**: AWS Lambda is a compute service that runs your code in response to events and automatically manages the compute resources, making it the fastest way to turn an idea into a modern, production, serverless application. We’ve used this cost-efficient service mostly for the backend processing.

**Amazon Rekognition**: Amazon Rekognition is a cloud-based image and video analysis service that makes it easy to add advanced computer vision capabilities to any application. The service is powered by proven deep-learning technology. We’ve used Amazon Rekognition and other services like Kafka for real-time data processing.

**Amazon Polly**: Amazon Polly is a cloud service that converts text into lifelike speech. It supports multiple languages and includes a variety of lifelike voices. We’ve used Amazon Polly for generating voice commentary.

**Amazon Transcribe**: Amazon Transcribe is an automatic speech recognition service that uses machine learning models to convert audio to text. You can use Amazon Transcribe as a standalone transcription service or add speech-to-text capabilities to any application. We’ve used Amazon Transcribe for extracting text from live commentary.

**Amazon Bedrock**: Amazon Bedrock is a fully managed service that makes high-performing foundation models (FMs) from leading AI startups and Amazon available for your use through a unified API. You can choose from a wide range of foundation models to find the model that is best suited for your use case. We’ve leveraged the power of Amazon Bedrock to facilitate our solution better.

**Amazon SageMaker**: Amazon SageMaker is a fully managed machine learning (ML) service. With SageMaker, data scientists and developers can quickly and confidently build, train, and deploy ML models into a production-ready hosted environment. We’ve leveraged the power of Amazon SageMaker to facilitate our solution better.

## Impact & benefits

### Impact

_SportsBuddy_ has a great impact on the sports community and the society. By addressing the needs of the blind and deaf community, _SportsBuddy_ promotes inclusivity and equality in the sports community. With the help of _SportsBuddy_, we can help blind and deaf individuals enjoy all types of sports to the fullest. It will uplift their moral spirits and also will provide them with a sense of belonging. They won’t feel left out anymore in the sports community. Our solution will help raise social awareness regarding the problems that blind or deaf individuals face in their day-to-day lives. It will inspire other communities as well to focus on the needs of people with disabilities.

### Benefits

Our solution is designed to benefit a wide range of stakeholders connected to sports, including fans, athletes, and sports organizations.

#### Fans

Our solution boosts the experience for fans to enjoy sports, particularly those with visual and hearing impairments. By providing accessible features such as real-time captioning, audio descriptions, sign language interpretation, etc, we ensure that all fans can enjoy sporting events to the fullest. This inclusivity helps grow a sense of belonging and engagement among fans, regardless of their abilities, enriching their overall experience and satisfaction.

#### Athletes

For athletes, our solution provides accessibility by ensuring the experience is the same and accessible to all fans, regardless of their sensory capabilities. By reaching a wider range of audience, athletes can amplify their impact and inspire individuals from diverse backgrounds. _SportsBuddy_ provides athletes with chances for greater exposure and recognition, contributing to their personal and professional growth within the sports industry.

#### Sports Organizations

Sports organizations can also benefit from our solution through increased audience engagement, brand loyalty, and revenue generation. By prioritizing accessibility and inclusivity, organizations can showcase their commitment to social responsibility and diversity, growing their reputation and appealing to fans and sponsors alike. Moreover, our solution may open up new revenue streams through subscription services, pay-per-view options, and targeted advertising, driving financial sustainability and growth for sports organizations.
Overall, our solution positively impacts users by promoting inclusivity, accessibility, and engagement within the sports community. By breaking down barriers and embracing diversity, we envision a future where sports experiences are amplified for all individuals, leading to positive changes in their overall enjoyment, participation, and operations within the sports industry.

## Commercial Viability

In a world where technology is evolving every day, ensuring inclusivity for all individuals, including those with visual and hearing impairments, is crucial. With approximately 253 million\[5\] visually impaired people and over 430 million\[6\] people with disabling hearing loss all over the world, there exists a significant market demand for products that address their needs.

![Recent statistics showing 253 million visually impaired and 430 million hearing impaired people]({{ sitePrefix }}/assets/img/articles/2024-05-22-Revolutionizing-Sports-Enjoyment-for-Sensory-Disabled-Individuals-in-MonstarHacks-2024/sportsbuddy-recent-stats.webp

<figcaption>Figure: Recent statistics showing visual and hearing impaired approximation</figcaption>

### Market Opportunity

This huge number represents more than just statistics; it points to an immense market opportunity waiting to be captured. Despite the growing number of individuals with visual and hearing impairments, this market remains almost uncaptured, presenting a huge opportunity for innovation and impact. Projections suggest that by 2050, the number of people with disabling hearing loss worldwide will surpass 711 million\[7\] and vision impairments by 588 million\[5\].

### Sports Engagement

Sports is considered a powerful platform for social engagement, and it's no different for the blind and deaf communities. No matter whether they are participants or spectators, a significant part of these communities actively engage in sports. But here's room to enhance their experience further. Imagine the impact of AI-based detailed commentary for blind audiences or real-time sign language translation for deaf audiences at sporting events. By providing these specialized experiences, we can increase their engagement and boost their enjoyment of sports, contributing to a more inclusive sporting environment for everyone.

### Forging Collaborative Partnerships

In our solution for visually and hearing-impaired individuals, we believe that collaboration is the key. Many government and non-governmental organizations are eager to support initiatives that can benefit these communities. By partnering with these organizations, we can have access to resources, expertise, and funding opportunities that can significantly uplift our efforts. These partnerships not only increase our reach but also demonstrate a collective commitment to improving the lives of individuals with visual and hearing impairments. We can take meaningful steps towards a more inclusive and accessible future.

## Challenges and Learnings

Our team set out on the hackathon with the dual goals of developing a solution that would be profitable and have a significant social impact. But this admirable goal wasn't without its difficulties, from figuring out the technological environment to understanding the business aspects of our solution.

### Challenge 1: Ideation and Social Impact

During the ideation phase, we targeted to find a hack that has both commercial viability and substantial social impact. We brainstormed, considering real-life problems around us.

**Solution**: Focusing on accessibility in sports enjoyment, we did some collaborative sessions and research and came up with this idea. Serving a neglected audience using our solution inspired us the most.

### Challenge 2: Technological Feasibility

At first, we were a bit unsure about the feasibility of our chosen technologies. We did some initial research and sorted it out with our mentor.

**Solution**: Our mentor, Iaroslav Ustinov, a solutions architect at AWS, guided us to choose the correct technologies. His expertise helped us assess the feasibility of our technologies.

### Challenge 3: Commercialization Strategy

At first, we were a bit unaware of how can we make this solution more commercially appealing to the judges and investors.

**Solution**: Peter Abdelkodos guided us, introduced some terms, and helped us to rethink the commercial viability of the solution.

## Conclusion

For our team, participating in the hackathon has been an enjoyable experience. We've overcome obstacles, welcomed creativity, and worked well together to develop a solution that has enormous potential to change society, from conception to implementation. Along with improving our technical proficiency, our journey has deepened our understanding of the value of collaboration and goal-driven creativity. Looking back on our hackathon experience, we're appreciative of the chance to make a difference and motivated to keep using technology to achieve important goals.

## Resources

- [1]: [Engage online sports fans with live event commentary using generative AI on Amazon Bedrock](https://aws.amazon.com/blogs/media/engage-online-sports-fans-with-live-event-commentary-using-generative-ai-on-amazon-bedrock/)
- [2]: [Leveraging AWS Bedrock to Revolutionize Live Sports Commentary](https://medium.com/@dobeerman/leveraging-aws-bedrock-to-revolutionize-live-sports-commentary-e03a7f7e7a85)
- [3]: [HandTalk Avatar](https://www.handtalk.me/en/)
- [4]: [AWS re:Invent 2023 - How to build generative AI–powered American Sign Language avatars](https://www.youtube.com/watch?v=LZgZ4K3OL80&ab_channel=AWSEvents)
- [5]: [NIH - World blindness and visual impairment](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5820628/#:~:text=In%202015%2C%20there%20were%20an,blind%20and%202.95%25%20have%20MSVI)
- [6]: [WHO - Deafness and Hearing Loss](https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss)
- [7]: [Projected number of people with disabling hearing loss worldwide in 2019, 2030, 2040, and 2050](https://www.statista.com/statistics/888569/number-of-people-with-hearing-loss-worldwide-projections/)

_Article Photo by [Chaos Soccer Gear on Unsplash](https://unsplash.com/photos/o8G05bDqYf0?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)_


Bridging the Gap: Revolutionizing the Sports Enjoyment for the Sensory-Disabled Individuals with AWS-Powered SportsBuddy


A major UK Healthcare provider commissioned Monstarlab to elevate the Contact Centre experience for an NHS-funded Patient Transport Advice Centre (PTAC), acknowledging the critical need for a digitally-enabled transformation to modernise and broaden service delivery via multiple digital channels.

The main drivers were the need to improve efficiency of staff to offset a reduction in resource, reduce average wait time for patients and replace existing disparate legacy systems which were lacking support.

The Monstarlab team conducted a discovery phase, laying the groundwork for a compelling business case to pilot the selected platform. Upon demonstrating an improvement over current technology, the decision was made to rollout Amazon Connect to the PTAC service.

![]({{ sitePrefix }}/assets/img/articles/2024-05-17-Amazon-Connect-Contact-Center-For-Patient-Transport/AmazonConnect.webp)

## Amazon Connect Features for Pilot Phase

Identified features to explore during the pilot phase were as follows:

### Telephony

The incumbent telephony system provided no queue information or expected wait times to users and suffered from frequent system disconnects if no active agents were available.

Amazon Connect offers the ability to configure Comprehensive IVR with better call routing, handling and queuing. We were also able to enable callback requests along with a separate callback queue with opening hours to ensure callbacks were handled at low peak usage times and within normal operating hours.

### Real-time and historical metrics dashboards and reports

Obtaining reports from the incumbent system took days or weeks and required manual mapping between separate systems which meant any insights may be up to a week old.

Amazon Connect offers real time and historical reports with data visualisation which provided the ability to manage staff scheduling and feedback in real time without manual intervention. The available metrics by default also provided additional insights which simply weren't possible before.

![]({{ sitePrefix }}/assets/img/articles/2024-05-17-Amazon-Connect-Contact-Center-For-Patient-Transport/analytics.webp)

### Contact Lens

In addition to the improved reporting, Contact Lens also offered real-time contact center analytics and quality management powered by natural language processing. This included sentiment analysis and the ability to categorise calls based on the content of the call without the need to manually review the call at a later date. These insights were only possible with significant manual resource investment previously which would only be available to resolve significant issues. By utilising contact Lens further, similar issues could be prevented or prioritised in real time.

![]({{ sitePrefix }}/assets/img/articles/2024-05-17-Amazon-Connect-Contact-Center-For-Patient-Transport/sentiment.webp)

## Outcome of Pilot Phase

Through the delivery of Amazon Connect handling all phone lines and agents for a period of 3 weeks, the following improvements were realised:

1. Average waiting time reduced by 45%
2. Calls serviced per agent increased by 24%
3. Average contact talk time reduced by 5%

Along with drastically improved management information visibility and agent satisfaction it was decided to roll out Amazon Connect to the full live service to replace the existing telephony.

## Next steps and Additional Amazon Connect Features

During the pilot phase, the following additional services provided by Amazon Connect were to be considered and discussed for future phases:

### Omni-channel customer experience

The current service is only available during office hours by phone.

Amazon Connect Flows can be utilised to offer patients their channel of choice, while saving them time and effort. Telephony and Website Chat Widget is provided out of the box but integrations with other chat clients, (such as Messenger, Slack, Teams etc). SMS, Email can also be configured and utilised to provide out of hours support and signposting.

### Amazon Lex

Limited Patient Data is provided "up-front" to the Agent handling the contact.

By utilising Amazon Lex, an ML Powered conversational engine, pre-eligibility questions could be provided to pre-populate case details for the agent so the beginning of a call isn’t spent gathering details. This will enable the agent to move straight to the eligibility assessment phase to provide a more streamlined service

### Amazon Customer Profiles and Cases

The current service tracks customer cases in a separate system to that handling the telephony.

Utilising Amazon Connect Profiles and Cases would allow agents to track and manage customer issues including those which require multiple interactions, follow-up tasks, and teams in the contact center. Agents can document customer issues with all the relevant case details, such as date/time opened, issue summary, customer information, and status, in a single unified view.

### Guides

The current eligibility assessment is completed in an in house system by an agent, supported by documentation in several formats along with team support via chat.

By implementing and utilising Step by Step Guides and Flows for the agents, they can be guided through the process during the call to populate the Amazon Connect case with the outcome.

### Amazon Q

In addition to the Step by Step Guides, real time information and assistance based on the content of the contact can be supplied to agents from existing knowledge repositories.

![]({{ sitePrefix }}/assets/img/articles/2024-05-17-Amazon-Connect-Contact-Center-For-Patient-Transport/amazonq.webp)

### API integrations and Tasks

API Integrations with 3rd party systems and APIs can be provided via Lambda functions within Flows, or the utilisation of Amazon Connect Tasks to automate agent tasks such as emails or bookings.

### Forecasting and Scheduling

Real time forecasting, capacity planning, and scheduling capabilities can be enabled to efficiently forecast or plan resources automatically based on historical metrics.

_Article Photos from [Amazon](https://aws.amazon.com/connect/)_


Implementing Amazon Connect to deliver an optimised Patient Transport Advice Service


In the dynamic world of sports, the connection between fans and their favorite teams is more than just fandom. It’s an emotional roller-coaster, a shared journey of success and failure. Because we saw such commitment was often overlooked by sports organizations, our team Argonauts participated in [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks) 2024 challenges with a mission to transform fan engagement. Our solution not only rewards superfans for their dedication but also fosters positive online communities by providing valuable insights to teams through our comprehensive dashboard.

## How we envisioned TALOS

We believe that by gamifying fan engagement, we can encourage and reward supporters for their unwavering support for their club. Through our platform, followers earn points for various interactions such as social media engagement, event attendance, and participation in club-related activities. These points act as currency, offering fans exciting prizes ranging from team jerseys and caps to the most desired tickets to the finals, as well as the chance to meet their favorite players which we think is the ultimate reward for a fan.

But our innovation doesn’t stop there. In today’s digital age, social media has become an integral part of the sports fans's lives. However, it’s not uncommon for this to turn toxic and spoil the spirit of the game. That’s where our solution steps in with its unique characteristic, it uses data from social platforms to identify and prevent toxic behaviour. By implementing a system that deducts points for toxic behavior, we aim to create a healthy and inclusive online community by biasing towards positive attitudes and encourage true sportsmanship among the followers.

For sports teams, understanding fan behavior and emotions is essential for improving their strategies both on and off the field. That’s why we created a robust dashboard that gives teams access to valuable insights gained from fan interactions. From monitoring engagement metrics to monitoring sentiment analytics, teams gain a holistic understanding of their fan base, allowing them to tweak their marketing efforts, improve fan experience, and deepen their relationships with their supporters.

Essentially, our solution is a win-win for all stakeholders. Followers are rewarded for their fandom and teams gain valuable insights of their fans and business.

## Technical architecture insights

The architecture that supports our solution is serverless, which enables a pay-as-you-go service model. This approach will prove highly cost-effective in the long term as demand increases for our app. We utilize AWS Fargate to seamlessly deploy our three services. These services connect our app with social media platforms, extract data from social media posts, and securely store it in an S3 bucket. Additionally, AWS Step Functions orchestrate the smooth processing of steps for our AI/ML tools, ensuring efficient and reliable execution of tasks.

At the core of our architecture, we embrace the cutting-edge capabilities of three powerful AI/ML services offered by AWS. Firstly, AWS Personalizes categorizes sports fans into loyal and seasonal groups, allowing for tailored recommendations based on their preferences. Next, Amazon Comprehend analyzes social media posts to understand fan sentiments, entities, and key phrases, providing valuable insights into fan emotions and preferences. Lastly, Amazon Rekognition deciphers visual content from images and videos shared by fans, identifying objects, people, text, scenes, and activities, enriching our understanding of fan engagement. Together, these services form the foundation of our solution, enhancing the overall fan experience in the APAC region and beyond.

As we will analyze large sets of data that are coming from the users' social media accounts, we need to ensure every piece finds its perfect place. To generate precise insights we will collect, extract, and organize the raw data using S3 Bucket, AWS DynamoDB, AWS Glue, AWS Athena and AWS RDS.

Last but not least, to ensure that our analysis is thorough and insightful, we employ AWS Location Service to pinpoint regions with the highest social media activity, helping us target specific demographics effectively. Additionally, AWS QuickSight provides visual data analytics for our B2B services, enabling us to gain valuable insights and make informed decisions based on the data collected and analyzed from social media platforms.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/MainDiagram.webp>
<figcaption>Main Architecture Diagram</figcaption>
</figure>

Let's simplify our large AWS diagram by breaking it into smaller pieces for easier explanation.

### User Authentication And Data Extraction

In this step, users sign up using AWS Cognito for authentication. Then, they answer questions to identify their favorite club/league, with AWS Personalize guiding the process. Next, users add their social media accounts to our app, which is managed by AWS Fargate. After adding their social media accounts, AWS Personalize again analyzes users' social media activity to identify their true fan identity and develop personalized recommendations. Additionally, another AWS Fargate service fetches public data from users'social media accounts and saves the data to an S3 bucket for further use.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/Diagram-2.webp>
<figcaption>Diagram 2</figcaption>
</figure>

### User Social Media Post Analysis

In this step, AWS AI/ML services play a big role. Two services run together: one analyzes users' social media posts with AWS Comprehend, while the other analyzes users' photos/videos with AWS Rekognition. AWS Step Functions coordinate these analyses, with multiple Lambda functions executing based on our algorithm. Additionally, AWS Location Services identify the location of social media activity. After analyzing and extracting all the data, it's saved in AWS DynamoDB.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/Diagram-3a.webp>
<figcaption>Diagram 3(a)</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/Diagram-3b.webp>
<figcaption>Diagram 3(b)</figcaption>
</figure>

### Analyze data in DynamoDB with Amazon Athena Federated Query

Since we're dealing with various data sources and are uncertain about the scale of analytics needed, it's wise to opt for a lightweight querying solution. Athena comes into play here, using data source connectors running on AWS Lambda for federated queries. These connectors translate between our data sources and Athena, automatically detecting metadata before each query. Additionally, we can use the AWS Glue Data Catalog to supplement metadata for connectors, especially useful when a data source lacks its metadata source, like DynamoDB. To share analysis results across teams, a custom dashboard is created using AWS Quicksight

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/Diagram-4.webp>
<figcaption>Diagram 4</figcaption>
</figure>

### Frontend
In this setup, we're using three databases: two AWS DynamoDB and one AWS relational database. These databases are connected to auto scaling-enabled EC2 services through an API Gateway.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-16-Revolutionizing-Fan-Engagement-Introducing-Talos-from-Team-Argonauts/Diagram-5.webp>
<figcaption>Diagram 5</figcaption>
</figure>

_Article Photo by [Jimmy Connover](https://unsplash.com/photos/people-watching-baseball-SEQ2VI0KI6A)_


Revolutionizing Fan Engagement: Introducing Talos from Team Argonauts


[MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), the annual competition hosted by Monstarlab, has reached its 4th edition, bringing together talented individuals from diverse backgrounds who are driven by innovation and dedicated to creating a positive social impact. This year, the challenges are brought to the table with collaboration with our partner cloud provider AWS.

## Challenge statement

Our challenge was to transform fishing matches with technology in order to streamline organization, creating immersive watching experiences while collecting valuable insights based on real data analysis for spectators, fishermen, sponsors, advertisers and equipment producers.
Fishing competitions involve anglers competing to catch the largest, heaviest, or most fish within a set time, often requiring strategic planning and keen understanding of aquatic environments. Competitions can take place in a variety of settings, from tranquil freshwater lakes and rivers to the vast and challenging environments of deep-sea fishing. These events are open to a wide range of participants, including amateur local anglers, seasoned professionals, and anyone in between looking to test their skills.
However, these events can face challenges in fan engagement and operational efficiency such as paperwork, complexity of live data recording, overall event organization and others.

## The proposed solution

We want to perform a digital transformation of such fishing competitions by building a platform that targets to solve the challenges by bringing interactable viewing experience and data analysis for competitors, equipment manufacturers and fishing enthusiasts. Event management will get digitized, enhancing organization and sponsor interactions, helping in building digital communities to connect fans more deeply with the sport.
This approach will not only streamline the logistics of hosting competitions but also will transform how spectators interact with and enjoy these events.

### Beneficiaries of the Platform

The main benefits include the following:

- Event Organizers benefit from streamlined event management.
- Spectators enjoy a more immersive and interactive viewing experience.
- Competitors gain valuable insights from post-event analytics, helping them refine their techniques.
- Sponsors and Advertisers engage more effectively with a captive audience through targeted advertising.
- Equipment Manufacturers use detailed usage data as a feedback to improve their products.

### Advanced Viewing Experience

For spectators, the experience is elevated with multi-angle live streams and interactive features. Multi-camera setup, including underwater cameras, will allow users to choose from various camera feeds, including those controlled by producers for dynamic shots and others that can be selected by viewers.
Configurable overlay will allow users to view current match statistics, records and other information that will help to understand the current situation on a deeper level.
All these ensure that every significant moment of the competition is captured and accessible, bringing viewers closer to the action than ever before.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-12-Transforming-Fishing-Matches/platform-overview.webp>
<figcaption>First look at the ReelStream app.</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-12-Transforming-Fishing-Matches/features-overview.webp>
<figcaption>Competitors list, available camera list, chat and other features.</figcaption>
</figure>

### Data Analysis and Utilization

The platform employs algorithms to analyze data collected during competitions. This analysis includes the effectiveness of different equipment, the strategies used by competitors, and environmental impacts on fish behavior. During a match, spectators will be able to view live data. After the match is done, relevant stakeholders will have access to final data analysis, helping them make informed decisions to enhance performance.
This data can be provided to the manufacturers, so they can improve their professional products.

### Promoting Sustainability and Education

Our platform also emphasizes sustainability and education by providing viewers with information on responsible fishing practices and the environmental aspects of fishing. This helps to promote a more sustainable approach to the sport and educates the public about the importance of conservation in fishing environments.

## Our Tech stack
- Machine Learning and AI using AWS Web Services
- React JS (Web) & Flutter for Desktop (MacOS, Windows, Linux) and Mobile (iOS and Android) Apps
- Python (Backend)

### High level AWS Infrastructure diagram and description

In the diagram below you can see the high level AWS architecture with the main services leveraged in our solution.
- S3 buckets to store raw and normalized data
- Lambda functions to trigger other AWS services like AWS Rekognition/SageMaker
- AWS Elemental MediaLive for normalising different feeds from various cameras setups
- AWS Elemental MediaPackage for real time packaging and streaming videos to end users
- AWS Interactive Video Service (IVS) for enabling interaction with streamed videos securely to end users
- AWS Rekognition (Custom Labels) with AWS Sagemaker to provide a suitable and efficient ML models for input raw data analysis
- AWS RDS for storing the data analysis results for further usage
- AWS Step Functions for orchestrating the over all flow of the platform

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-12-Transforming-Fishing-Matches/non-interactive-devices.webp>
<figcaption>AWS Architecture for non-interactive devices</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2024-05-12-Transforming-Fishing-Matches/interactive-devices.webp>
<figcaption>AWS Architecture for interactive devices</figcaption>
</figure>

## Conclusion
 We believe that the overall interest in fishing matches will increase thanks to our solution, which will lead to fulfilling our vision and mission values.

_Article Photo by [Stephen Momot](https://unsplash.com/photos/six-black-and-yellow-fishing-rod-in-boat-UivGzIDhVyw?utm_content=creditShareLink&utm_medium=referral&utm_source=unsplash)_

_Screenshot from a video by [Milliken Fishing](https://www.youtube.com/watch?v=HpXAbrjVCyw)_


Digitalizing Fishing Competitions, the journey of The ReelStreamers team in MonstarHacks 2024


In this article, we will discuss how to set up SonarQube with Docker and produce reports on the code quality of your projects.

## Install Docker on your system

Install Docker into your system using any of the guides online for your preferred system.

Run the following command in your terminal to verify docker has been installed.
```sh
docker -v
```

## Installing SonarQube on Docker

To run SonarQube on your machine using docker, first open a new terminal tab and then pull the image from docker hub using the command below.
```sh
docker pull sonarqube
```

Once the image has been pulled, start the container by running:
```sh
docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest
```

Once your instance is up, log in to the Sonarqube dashboard by navigating to [http://127.0.0.1:9000](http://127.0.0.1:9000) and using System Administrator credentials:
  * login: admin
  * password: admin

## Create a new project on Sonarqube

After logging in, you should see this screen.
![]({{ sitePrefix }}/assets/img/articles/2024-02-06-Setup-Sonarqube-with-Docker/sonarqube-intro.webp)

  * Let's select manually.
  * Enter your project display name and main branch name.
  * Select a baseline for new code for your project
  * Your project should now be successfully created in Sonarqube

## Analyze your project on Sonarqube

After creating your project, you should be seeing this screen.
![]({{ sitePrefix }}/assets/img/articles/2024-02-06-Setup-Sonarqube-with-Docker/sonarqube-analyze.webp)


Since we are going to analyze our project locally, click on Locally. Then generate your project token. Select the appropriate options under Run analysis on your project.

Let's also run SonarScanner via docker, no need to download and install it.

First create a file in your project in the root directory called
**sonar-project.properties**

```
# must be unique in a given SonarQube instance
sonar.projectKey={projectKey}

# --- optional properties ---

# Encoding of the source code
sonar.sourceEncoding=UTF-8

# Here, you can exclude all the directories that you don't want to analyze.
# As an example, I'm excluding the node_module and .next directories
sonar.exclusions=node_modules/**, .next/**
```

Then, to run the SonarScanner and scan your code, navigate to your project in a new terminal tab and execute the following command:

```sh
docker run \
    --rm \
    --network=host \
    -e SONAR_HOST_URL="http://127.0.0.1:9000" \
    -e SONAR_SCANNER_OPTS="-Dsonar.projectKey={Project key}" \
    -e SONAR_TOKEN="{authenticationToken}" \
    -v "$(pwd):/usr/src" \
    sonarsource/sonar-scanner-cli
```

Eg:
```sh
docker run \
    --rm \
    --network=host \
    -e SONAR_HOST_URL="http://127.0.0.1:9000" \
    -e SONAR_SCANNER_OPTS="-Dsonar.projectKey=hrbot" \
    -e SONAR_TOKEN="sqp_9f5ff08eba566ab37afe5f36104fcxxxxxxxx" \
    -v "$(pwd):/usr/src" \
    sonarsource/sonar-scanner-cli
```

After the scanning is complete, go back to the dashboard and you should see the following screen:
![]({{ sitePrefix }}/assets/img/articles/2024-02-06-Setup-Sonarqube-with-Docker/sonarqube-result.webp)

You have now completed the scanning of your code for the first time. You can now go ahead and solve any issues and smells that are reported by Sonarqube on your path to a sparkling clean code!

To turn off the Sonarqube container afterwards you can run the command:
```sh
docker stop sonarqube
```

To turn it on again for later use, you can run the command:
```sh
docker start sonarqube
```

_Article Photo by [Milad Fakurian](https://unsplash.com/photos/PpgY7sjpf_0)_


Setup Sonarqube with Docker

## Introduction

Single Page Applications (SPA) revolutionized the web development landscape and provided users emulation of native applications. With it however, also came added complexity. “Meta frameworks” such as NextJS or NuxtJS further add on to the complexity as developers are now tasked with managing complex interactions between server and client. In an attempt to reduce this complexity, [HTMX](https://htmx.org/) has re-emerged into the scene.

What was formally called “[intercooler.js](https://intercoolerjs.org/)” is now HTMX. HTMX in a nutshell is a small client side Javascript library that attaches attributes to DOM elements. These DOM elements are then used to send requests to the server. What this effectively allows the developer to do, is to emulate SPA reactivity by controlling it from server by sending small, HTML “snippets”, which then HTMX swaps or inserts somewhere in the DOM. This simple concept is enough to power many web applications, which do not require the large baggage that modern frontend frameworks come with.

Among many other benefits, this also reduces the need for managing state in two places (client and server), and also allows developers to code in any backend language they prefer.

In this post I’d like to highlight HTMX's simplicity by building a small todo list application together while discussing its benefits and why one should consider using it in their next projects.

## A Todo List

The best way to understand HTMX is by building something with it. As is tradition, let’s build a todo list application with it. Here already we start with the inherent merit of HTMX and that is that we can use it together with any programming language of ones choice. This allows teams not to be locked into some framework that requires specialized infrastructure. In fact, many developers may choose HTMX simply to get away from JavaScript, and we’ll be doing just that with Python and Flask.

This combination offers a wonderful experience for web development, enabling the creation of web applications that emulate Single Page Applications (SPAs) with ease and efficiency. With Flask and HTMX, you can avoid the complexities associated with traditional build systems like webpack, while effortlessly managing state between the client and server. This approach is also ideal for developing proof of concepts at high velocity. We’ll do this with the coding platform Replit. If you wish to follow this on your local machine, please refer to this [tutorial](https://flask.palletsprojects.com/en/2.2.x/quickstart/), but since the point of this article is to display HTMX’s ease of use, we will disregard application setup.

You can start right away with Replit by navigating to a new [Repl](https://replit.com/new) and selecting the Flask template.

Let’s start right await by making an index template that wraps our entire page:

```html
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Todo App</title>
    <script src="https://unpkg.com/htmx.org"></script>
    
    <script src="https://cdn.tailwindcss.com"></script>
  </head>
  <body class="display flex flex-col items-center w-full relative h-screen">
    <h1 class="text-3xl flex gap-4">
            {% include 'star.html' %}       Today    
    </h1>
    <div id="todo-list">
      
    </div>
    <form hx-post="/add-todo" hx-target="#todo-list" hx-swap="“innerHTML”">
      <button
        type="submit"
        class="text-white text-3xl absolute right-5 bottom-5 rounded-full bg-blue-600 aspect-square w-6"
      >
        +
      </button>
    </form>
  </body>
</html>
```

And the following Python file that will contain all our logic:

```python
# main.py
app = Flask(__name__)

@app.route('/')
def index():

  return render_template('index.html')
```

On the page we simply include HTMX via a script tag, no build step required.

Our first introduction to HTMX begins with the `hx-post`, `hx-target`, and `hx-swap` attributes on the `form` element. HTMX empowers web developers by allowing the addition of special attributes to any standard HTML element. These enhanced elements gain the ability to initiate HTTP requests. The responses in the form of HTML fragments, can then be dynamically inserted or replaced in any part of the webpage. This feature introduces a seamless way to update content without needing a full page refresh. In our case, we want to put our tasks into the `div` element with the id of "todo-list".

Let’s do this by inserting an empty task when the user presses the “+” button. We start by adding an endpoint that will create the HTML fragment, and then return that fragment:

```python
# main.py
@app.route('/add-todo', methods=["GET"])
def task():
  return render_template('task.html')
```

And the following template:

```html

<div class="bg-gray-800 rounded-lg p-4 max-w-lg mx-auto my-6">
  <div class="flex items-center">
    <input
      type="checkbox"
      class="form-checkbox h-5 w-5 text-yellow-400 rounded border-gray-600"
    />
    <input
      type="text"
      placeholder="New To-Do"
      class="ml-4 bg-transparent border-0 placeholder-gray-400 text-white focus:outline-none"
      style="width: calc(100% - 2rem);"
    />
  </div>
  <textarea
    placeholder="Notes"
    class="mt-2 bg-transparent border-0 placeholder-gray-400 text-white focus:outline-none w-full"
  ></textarea>
  <div class="flex items-center justify-between mt-4">
    <button class="text-yellow-400 focus:outline-none">
                  Today        
    </button>
    <div class="flex space-x-2">
      <button class="focus:outline-none">
                      {% include '/svg/line.html' %}            
      </button>
      <button class="focus:outline-none">
                        {% include '/svg/dropdown.html' %}            
      </button>
      <button class="focus:outline-none">
                        {% include '/svg/hamburger.html' %}            
      </button>
    </div>
  </div>
</div>
```

With some added styling, the actual important points of this fragment is the `input` elements which will play a part soon. Clicking on the + button will now display a new, empty task. Let’s see how this works. The button looks like this:

```html
<form hx-get="/add-todo" hx-target="#todo-list" hx-swap="“innerHTML”">
  <button
    type="submit"
    class="text-white text-3xl absolute right-5 bottom-5 rounded-full bg-blue-600 aspect-square w-6"
  >
    +
  </button>
</form>
```

The special attributes added by HTMX `hx-get`, `hx-target` and `hx-swap` tells us, “when the form is submitted, issue a (hx-)get request, and (hx-)target the element with id “todo-list, and (hx-)swap the inner content of this HTML.”. The concept of emulating SPA-like behavior via attributes is what forms the entire behavior of HTMX. Here’s a few other attributes which would allow us to do other things:

1. `hx-trigger` would allow us to trigger requests based on behavior, such as on mouse hover.
2. `hx-post` sends a post request instead of a get request. You can also do this with PUT, DELETE, and PATCH.
3. `hx-boost` allows us to convert all anchor tags and forms into AJAX requests that, by default, target the body of the page.
4. `hx-push-url` allows us to interact with the browser history API.

And many others. This also does not include the long list of values for each of these attributes which further allows us to customize behavior. It’s also important to note that these attributes can be applied to any element. These are all well documented in the [documents page](<[https://htmx.org/docs](https://htmx.org/docs)>).

Let’s improve the todo list application by adding attributes to append tasks to the list, and to edit the individual tasks. Instead of swapping the entire inner HTML of the `#todo-list` element, we can append to the end with `beforeend` as such:

```html
<form hx-get="/add-todo" hx-target="#todo-list" hx-swap="beforeend"
  …
 
```

These tasks are ephemeral, so we need to start adding some state. Since HTMX allows us to control the front end from the server side, we don’t need to reach for state synchronization solutions as it is often needed in applications that use larger frontend frameworks. We’ll drive everything from the server as our source of truth:

```python
# main.py
task_list = [{
    "id": 222,
    "title": "Schedule doctors appt",
    "notes": "Make sure its in the evening after the 14th."
}]


@app.route('/todos', methods=["GET"])
def load_task():
  return render_template('tasks.html', tasks=task_list)
```

We load our state into the template, and further split up our code into fragments, similar to components (create a file `templates/tasks.html`):

```html
<ul id="todo-list">
      {% for task in tasks %}         {% include 'task.html' with context %}    
  {% endfor %}
</ul>
```

The individual item templates are ready to consume the data supplied by Flask. This completes our "READ" process in our little CRUD application. Since we already have the "CREATE" process from our first code written, we can continue here onto our "UPDATE". We can add the ability to edit a task with a nice UX pattern. The user should be able to just type and the tasks will be updated seamlessly. We can do this by debouncing on key input. This can typically be quite terse with other frontend frameworks, requiring several lines and some intricate state updates, but with HTMX it is elegant:

```html
<form
  id="task-form-{{ task.id }}"
  hx-post="/update-todo"
  hx-trigger="keyup from:input, keyup from:textarea delay:500ms
"
>
  <div class="flex items-center">
    <input type="hidden" name="task_id" value="{{ task.id }}" class="" />      
      <input type="checkbox" class="“…”" name="checked" />        
    <input
      type="text"
      name="title"
      placeholder="New To-Do"
      class="“…”"
      value="{{ task.title if task.title }}"
    />
  </div>
  <textarea name="notes" placeholder="Notes" class="“…”">
{{ task.notes if task.notes }}</textarea
  >
</form>
```

The `hx-trigger` on top of the form does a lot of work for us here, and it almost reads like in plain English, “trigger on key from input or textarea, and delay for 500 milliseconds”.

Finally, we can update the data on our backend:

```python
@app.route('/update-todo', methods=["POST"])
def update_todo():
  # Extract the data from the form
  task_id = request.form.get('task_id')
  title = request.form.get('title')
  notes = request.form.get('notes')
  checked = request.form.get('checked') == 'on'

  # Update the task in the list with some fun list comprehension
  global task_list
  task_list = [{
      **task, 'title': title,
      'notes': notes,
      'checked': checked
  } if str(task['id']) == task_id else task for task in task_list]

  # return nothing so to disallow default swapping from HTMX
  return '', 204
```

And with that, we've concluded our "UPDATE" process code. I leave the exercise of deletion and sorting (or whatever else you may think of) to the reader. If you’ve been convinced by now, you can probably already see how trivial it can be to add the rest of these features.
![Github Branches]({{ sitePrefix }}/assets/img/articles/2023-12-19-Introduction-to-HTMX/img-1.webp)
Up until now we've explored how to do some simple CRUD operations. One reason however that people chose to use the complex modern frontend frameworks is the ability to have state dependent on other state update automatically. While HTMX may not be the best approach here when requiring a complex set of dependencies across many DOM elements, it is still absolutely achievable to accomplish this with HTMX. HTMX provides the [`hx-swap-oob`](https://htmx.org/attributes/hx-swap-oob/) attribute that lets you swap outside of the target element. As the HTMX documentation says:

> This allows you to piggy back updates to other element updates on a response.

With this in mind, we can add a great example for our little app. What we will add is a counter at the top of the application header. This counter is a state that is dependent on the number of todo items. When we add a new todo item, we can update the top header with the new updated count value:

At the end of the `task.html` file we add:

```html

{% if new_task_length %}
<span id="counter" hx-swap-oob="true">{{ new_task_length }}</span>
{% endif %}
```

And then we add the following item to the return statement of the `/add-todo` endpoint:

```python
@app.route('/add-todo', methods=["GET"])
def task():
# ...rest of logic
	return render_template('task.html', task={}, new_task_length=len(task_list))
```

this demonstrates how easy it is to manage state dependent updates via HTMX's `hx-swap-oob` feature. Here, with just a few lines of code, we have created an additional feature in our app - a counter that automatically updates based on the current length of our task list. The length of the task list is a dependent state, reflecting the total count of todo items. When a new task is created, the counter in the header is automatically refreshed to display the new count.

I want to conclude this tutorial exercise section by discussing animations, one reason why many people choose to use Javascript frameworks.

![Github Branches]({{ sitePrefix }}/assets/img/articles/2023-12-19-Introduction-to-HTMX/img-2.webp

Indeed, animations are a key aspect of modern web applications, often being a primary reason for choosing JavaScript frameworks. However, with HTMX, you don't have to forgo this essential element of user experience.

### Adding Animations with HTMX

Animations can be achieved through various methods while using HTMX. A significant portion of interactive feedback can be handled with CSS. Modern CSS provides enough for creating animations and transitions, making it possible to animate changes in your UI in response to HTMX events.

For instance, using CSS transitions, you can smoothly animate the appearance and disappearance of elements, or their transformation in response to user interactions. This can be particularly effective for operations like adding, updating, or deleting items in a list, where visual feedback enhances the user experience. With HTMX, this is taken a step further with great integrations such as swap transitions. Take an example from the docs:

> If you want to fade out an element that is going to be removed when the request ends, you want to take advantage of the `htmx-swapping` class with some CSS and extend the swap phase to be long enough for your animation to complete. This can be done like so:

```html
<style>
  .fade-me-out.htmx-swapping {
    opacity: 0;
    transition: opacity 1s ease-out;
  }
</style>
<button
  class="fade-me-out"
  hx-delete="/fade_out_demo"
  hx-swap="outerHTML swap:1s"
>
  Fade Me Out
</button>
```

### Leveraging JavaScript Libraries

For more complex animations, you can integrate lightweight JavaScript libraries. GreenSock (GSAP) is an excellent example of a library that provides advanced animation capabilities with a simple API. By combining HTMX with such libraries, you can create sophisticated animations that bring your UI to life, without the overhead of a full JavaScript framework.

Lastly, I'd like to point out the emerging Transitions API, currently only available in Chrome. This API is set to offer native support for smooth transitions, potentially reducing the reliance on external libraries for such effects, thus making the necessity of SPA frameworks less relevant. HTMX too, integrates well as it provides access to this API via the `hx-swap` attribute, which you can learn more about [here](https://htmx.org/examples/animations/#view-transitions).

## Wrapping Up

HTMX offers a sublime experience for developers with simplicity and functionality. It allows for developers to focus on business logic without compromising on UI/UX. With HTMX, basic REST knowledge is enough to build a fully functional user interface, reducing complexity, development time, and costs.

HTMX isn't a solution for everything, though. For more complex applications, a comprehensive JavaScript framework may be more suitable. The state management and component-based architectures that HTMX doesn't inherently support might be a better fit depending on the application and needs.

Despite this, HTMX's ease of integration and minimalistic approach makes it a valuable tool that should always be considered. It's particularly effective for enhancing server-rendered pages or for projects where a full JavaScript framework is unnecessary. Ultimately, HTMX stands out for its ability to deliver efficient, interactive web experiences with minimal overhead, making it a technology worth considering in various web development scenarios.

Since there is always more to learn, please consider checking out the [official documentation](https://htmx.org/docs/) and the [essays](https://htmx.org/essays/) as well as the [free book](https://hypermedia.systems/) provided by the HTMX author. You can also visit the source code used for this project here: [Replit](https://replit.com/@hhofner/KnownHeartyApplescript)

## Resources

_Article Photo by [HTMX](https://htmx.org/)_

An Introduction to HTMX

Today, we're chatting with [Nikita Sirovskiy](https://www.linkedin.com/in/nivisi), Lead Mobile Engineer who on top of working on client projects, plays an important role in enhancing Monstarlab's Flutter Service Offering by driving meetings, creating & managing tasks and helping with people’s allocation.

Nikita is, in his own words, a "healthy perfectionist, stoic and a creative person". In his free time, he likes to play guitar, sing, write stories, poems and books. Nothing is published yet, but hopefully one day we'll get to read his work.

![Prague Team]({{ sitePrefix }}/assets/img/articles/2024-01-19-Interview-Nikita-Sirovskiy/team.webp
<figcaption>In the photo: Prague team (Omar Bekier, Ekaterina Surovtseva, Eleonora Perevozchikova, Juraj Longauer, Nikita Sirovskiy, Pedro Massango, Oleksandr Mochalov, Esmaeil Abedi, Muhittin Kaya)</figcaption>

_**Nikita, can you describe your journey of transitioning from a developer to a project lead? What were the most significant changes you had to adapt to?**_

“Extreme Ownership”, that’s what sums it up best. I've always been proactive and tried to enhance things on all the projects I was a part of. Questioning decisions made outside of my direct responsibility zone was and is very important to me. I work not only on mobile apps, I work on products. A right question at the right time might turn the tide. I know people say ‘I’m not getting paid for this or that.’, but I never understood this. Of course you must not do things for others, yet, if you see a room for improvement, why not just improve it?

One of the most significant changes is that now, most of the time, I am the person who answers questions, not the one who asks them. Deadlines, scopes, code reviews and all this. After all this, there’s not much time to code myself. While being a lead gives me a bird-eye view on the entire project, which is very interesting, sometimes I miss these days where all I had to do was just to code. Coffee, good music, quietness, flow state… Yes, sometimes I definitely miss this.

_**Q: During your first project as a lead, what were some of the biggest challenges you faced? Could you share a story where you had to overcome a difficult obstacle, and how you managed to do it?**_

Self-doubt. That’s the biggest thing for sure. It was hard to accept that now it’s me who is responsible for making decisions and saying yes or no to other’s decisions. Sometimes the pressure was high, sometimes I was not confident in my own solutions and decisions, sometimes the deadlines were really tight. Although people around me — my family, my line manager, my project manager and my teammates — supported me a lot. Not only with valuable advice, but also by saying that the result is fantastic so far. Another day, when looking at the result myself, I realized that we’ve built the best mobile app I ever worked on. It’s stable, developer experience is great, the code is clean, and the UI/UX is amazing. That gave me a lot of confidence and the game changed.

> “Don’t pull yourself backwards” — that’s what one of my friends told me.

_**Q: Given your experience with remote work, how do you handle the dynamics of leading a team remotely, especially when occasionally there is a need to visit client offices as well?**_

I worked remotely since the beginning of my career, and I still do. Although Monstarlab’s borderless approach took it to another level. Even people who work from office usually sit in different offices all around the world! This is amazing. You can meet so many different people. This is how we work on the current project as well.

We try to be more autonomous and contextualized. Less meetings, more documentation: in-code, Confluence, Google docs, or whatever is more convenient for a specific topic. Less direct messages in messengers, more communication in corresponding tools (GitHub, Figma, etc). Different time zones is a challenge itself. I want us to reach the point when, if someone needs something, they will be able to find it in the docs. So we won’t need to disturb or wait for each other in Slack or in a meeting room.

But working remotely means that colleagues are more disconnected from each other. Sometimes it’s hard to understand the needs or mood of your teammates, so periodic catch ups are very important. I have 1-on-1 calls where I sincerely ask my teammates about what is going on. Is there something to change, a problem that must be fixed or just a question that needs an answer. We find it very beneficial. I was given a lot of suggestions on these calls. It also allows you to discuss off-topic things which leads to better connection with a person. Although it is important not to turn these calls into rituals that you blindly follow just “because”.

Work trips are also very beneficial. All my visits were filled with productive and interesting time. Brainstorming sessions, lunches, morning coffees, debug sessions. This for sure enriches your connection with people. As someone who works remotely most of the time, I value it a lot. Perhaps one day I decide to relocate to be with the team all the time. I’m very curious how it would be.

_**Q: What advice would you give to developers who are about to take on their first leadership role? Are there any key lessons you wish you had known when you started?**_

Well, don’t pull yourself backwards, that’s for sure. You are where you are for a reason. Your skills and commitment have led you to this role, so just keep it up and do your best.

For more specific things, learn how to organize yourself. Self-organization helps to keep track of things: questions raised on calls or chats, project goals, tech debt, ideas etc. Proper tracking allows you to plan better, oversee everything and move faster. I started using Apple Notes extensively. I have folders for interviews, TODOs, suggestions, conversations, and basically anything that is important in any way. One thing I love is the new (maybe not that new anymore) feature with the quick note in the corner of your screen. I keep my TODO list there. If I need to note something, I just move my cursor to the right-bottom corner and start noting.

I write a lot of documentation. This is an extremely beneficial tool for time saving and optimization. We don’t need to re-explain rules and there’s less need to sync. If in doubt, one can open the documentation to learn how certain things are done.

Another thing is talking to people. This will sound classic, but everyone is very different, and everyone requires a different approach. You will need to provide constructive feedback, be a mentor for your colleagues, raise problems and concerns to managers and the business, suggest ideas and say no. The latter is a skill itself, a very important one. I like this phrase that I heard somewhere, «If it’s not a 100% yes, then it’s a no». Of course it’s not really applicable here (otherwise we would’ve been blocked most of the time!), but this idea is still a good foundation for making decisions. Remember, you’re in charge now, and raising concerns, whatever those are, is a must. I failed to do this a few times. Those problems came back at a later stage of the project. Nothing big, fortunately, yet still… I learned this very well.

_**Q: How has your experience been working at Monstarlab? Could you share some highlights or key moments that have shaped your journey here?**_

It’s been fantastic so far! Working here has been a journey of growth and challenges. What I can highlight is the level of trust that you have here. On my very first project, I quickly shifted to conducting demos, enhancing code practices, and doing code reviews. On my second project, when the lead relocated to a different time zone, I stepped up to handle most of the code reviews, demos and crucial feature finalizations. On my third project I was mostly solo, which made me the one who talks with designers, backend developers, clients and PMs.

> The atmosphere on every project was amazing. Ask anyone, and everyone will tell you that people in Monstarlab are special. They are always eager to chat, to laugh, to help and support.

Someone even invited me to their home, and we spent evenings talking about life. Someone drives me to and from the office every time I’m on a work trip, and we have these short morning and evening conversations.

_**Q: Reflecting back to the beginning of your career, what initially inspired you to start programming? Have there been any specific role models in the tech industry who have influenced or guided your path?**_

I have a very vivid memory of asking my father “Who creates computer games?” and him answering “programmers”. That was the initial inspiration. I wanted to be either a programmer or a taxi driver. Don’t ask me why. Today I don’t even have any clue about cars. Anyways.

It became serious when my father discovered that you can create custom game maps, or modes, for my favorite computer game, Warcraft 3. This involved coding, designing characters, and crafting stories and worlds – a mix of technical skills and creativity. I think I was 7 years old when he showed me that program, World Editor. I spent countless hours, days and nights, in trying to craft things. My mother also played a crucial role. She forced me to learn more and take additional classes, even when I didn’t want to. But in a soft way. I always had full freedom in experimenting what I want from this life and in learning whatever I think is needed.

But I didn’t become a game developer! Maybe one day, maybe one day. I was lucky to have that map-creating period in my life. It has built a solid foundation for problem-solving and creative skills that I use today.

_**Q: Every profession has its lighter side, and programming is no exception. Do you have a favorite programming joke or a meme that resonates with you or brings a smile to your face?**_

Yes! I wanted to find the picture, but I couldn’t. There's this meme comparing users enjoying a new feature to developers tangled in a mess of code trying to figure out how to develop a new feature without breaking the app.

While humorous, it shows the real challenges in coding. I've seen poorly organized codebases that looked like spaghetti. It's a reminder that we should never trade-off maintainability for delivering something new. Organization, requirements, documentation and optimal processes make development cycles efficient and apps cost-effective.

It improves another thing that is often ignored, developer experience. The happier the devs, the more interesting it is to work on a project, the better the project will be. I hope that that meme is not applicable for us. Hope that in our case both of the sides of the picture represent happy people.


Behind the screens: Interview with Nikita Sirovskiy, Lead Mobile Engineer at Monstarlab

Enhancing ETL Expertise: A Backend Engineer's Insight into Leveraging AI Tools for Python Development

Today, we're chatting with [Nairah Thaha](https://www.linkedin.com/in/nairah-thaha-0a37269a/), a skilled Immersive Technologies Engineer at Monstarlab. Nairah joined the Monstarlab team back in March 2023, bringing with her over 7 years of expertise as a Unity developer with a focus on Immersive Technologies (AR, VR and Metaverse).

Born and raised in Dubai, she indulges in a variety of activities that reflect her dynamic personality. From practicing calisthenics and aerial dance to enjoying video games, paddle boarding, and tackling escape rooms, Nairah loves to challenge herself. And when it comes to food, she will never say no to a delicious bowl of ramen.

![Ennis and Nairah]({{ sitePrefix }}/assets/img/articles/2024-01-11-Interview-Nairah-Thaha/nairah_ennis.webp)
<figcaption>In the photo: Ennis Elmudir and Nairah Thaha</figcaption>

_**Nairah, could you tell us about your experience developing immersive technology apps that use generative AI? Could you share a cool prototype you have worked on recently?**_

I’ve started incorporating generative AI into my projects as early as March 2023. When the trend took off, I immediately saw its value and got started on ideating and developing new experiences. The latest one I developed was to help train customer facing employees to deal with difficult client cases. In this scenario, the “customer” was a generative AI human that had a specific banking challenge that you needed to converse with to help solve. The conversation would be recorded and at the end you get an evaluation of how well you handled the situation – what you could have done better, statistics on the conversation as a rating from 1-10, etc. Because the “customers” in the prototype are generative AI, it’s very easy to change out their prompt to handle different situations or even industries.

_**What have been some of the most significant challenges you've faced when creating immersive tech experiences using generative AI? Could you share an example where you had to innovate or think creatively to overcome these challenges?**_

When I had started off back in March, the generative AI scene in Unity was still quite fresh and there weren’t any helpful tools yet. My earliest prototypes had a pretty dissociative issue which was a long delay between you speaking to a generative AI avatar and it responding back to you. This was due to the processes of speech to text, analyzing the question and generating an answer, and then transforming this text back to speech using generative AI to sound more human. As a workaround, I started processing smaller chunks of text at a time, in order to reduce this delay. Eventually, many helpful tools came to market, such as Inworld, that make this process easier to implement and without the speech delay I was facing.

_**In a field as rapidly evolving as immersive tech and generative AI, how do you keep your skills and knowledge up to date?**_

With new challenges and brief timeframes, I’m compelled to find the most efficient and latest technologies to help me develop them. It’s been very helpful to have the freedom to explore new creative solutions and develop them. The constant practice of doing this over the last year has really helped me stay up to date with the latest technology.

_**Looking forward, where do you see the integration of generative AI and immersive technologies heading? What are some advancements you anticipate in creating more engaging or realistic experiences?**_

I feel generative AI is already in a very good state and I trust it to continue to get better over time. Where I anticipate more improvement would be in the realm of more realistic human characters to work with. While the avatars look great, there are nuances in real-time facial experiences and mouth movements that I’d be excited to see improve in order to create better immersion.

_**How has your experience been working at Monstarlab? Could you share some highlights or key moments that have shaped your journey here?**_

I think it would be difficult to highlight only one particular moment. As a whole, the environment I’m in is set up nicely to foster innovation and push the boundaries of what is possible. I also have a great team I can depend on and I cross collaborate regularly with other departments. This helps us come up with some really great solutions.

> As an immersive tech engineer, it’s always important to be up to date with the latest in technological innovation and Monstarlab gives me the platform to do so.

_**If you had the chance to choose one programming superpower, what would that be and why?**_

Read documentation once and remember it forever 😊. However, I am super happy to see tools such as Unity Muse come out, which keeps all its documentation alive in a generative AI chat, similar to ChatGPT. It’s very handy and has made programming a lot more efficient.

_**What's on your current tech wish list? Any specific gadgets or software tools?**_

Definitely the Meta Quest 3 for its mixed reality capabilities. I’m also looking forward to trying the Apple Vision Pro soon!


Behind the screens: Interview with Nairah Thaha, Immersive Technologies Engineer at Monstarlab

Welcome to our interview series!

Today, we're chatting with Adam Mack. With over 15 years at the company, starting back at Fuzz Productions (a digital agency which was [acquired by Monstarlab in 2019](https://monstar-lab.com/global/ml-news/fuzz_join)), Adam is a Solution Architect or Technical Product Manager, depending on the day of the week!

Living in Massachusetts, USA, Adam enjoys life with his partner Molly and their cat, Cray-cray. When he's not tackling tech problems, he's into pinball, making and playing video games, creating music, and cooking. Let's dive into his world!

![Adam and Cesar]({{ sitePrefix }}/assets/img/articles/2024-01-03-Interview-Adam-Mack/adam_cesar.webp)
<figcaption>In the photo: Adam Mack and Cesar Aguilar</figcaption>

_**Adam, could you share with us the story of how you entered the field of generative AI? What was the inspiration that led you to focus on this particular area in technology?**_

I was fascinated by Google Deep Dream back in 2015, and the bizarre dreamlike images it could create. That’s when I first became aware of AI image generation as an academic field, and I’ve sort of kept watching the space since then. Fast forward to 2021, and the fairly quiet announcement of Dall-E 1 from OpenAI; I was showing every single person I know the landing page, saying things like “look! It can make an avocado chair out of nothing!!”. I was very excited about the possibilities, to say the least.

_**For those who are new to the concept of generative AI, how would you explain it in simple terms?**_

For friends and family who know nothing about computers, let alone neural networks, I usually refer (half joking) to the analogy of waiting for a million monkeys to accidentally write Shakespeare - what if we could train the monkeys over time to learn what words mean, and which words go next to each other more frequently than others? Even if they can’t fully read or write like you or I can, you’d start to see patterns emerge that resemble Shakespeare (or whatever input) more closely than the random noise you’d otherwise get. Now imagine you can give positive reinforcement to specific monkeys that do a great job of generating Shakespeare-like patterns; you’re part of the way toward visualizing a neural network at work.

_**What do you think are some of the most exciting trends in the field of generative AI? How do these trends influence your work?**_

The most exciting trend, hands down, is seeing how product teams everywhere are taking the “basic” building blocks of generative AI and layering them with the fundamental building blocks that make up their product, leading to new interaction patterns that would have been unimaginatively complex to engineer just a few years ago. Just opening up natural language as a viable user interface for software is a game changer - I think anyone frustrated with the limitations of last gen voice assistants knows how much unrealized promise there is in the ability for a user to naturally converse with a machine. Especially when it comes to the context of providing enhanced accessibility or representation to audiences that are either underserved or have accessibility challenges.

_**Looking ahead, where do you see generative AI heading in the next 5 to 10 years? What kind of advancements or changes do you anticipate?**_

I have so much hope for generative AI’s potential to give an artistic voice to those without the resources or ability to “compete” with commercial interests. Suddenly, if you have a dream for a movie or illustration, it’s no longer locked up in your brain if you lack the artistic ability or tools to realize it. I feel like that could be transformative for society in the same ways that the printing press was transformative, in giving a voice to those who previously lacked a voice, giving access to creating art to those who lacked access before. At the same time, especially as AI outputs become closer to commercially viable replacements for human labor, we’re going to see a huge backlash against these technologies and the use of these technologies in commercial applications. I don’t know who wins that battle for hearts and minds of consumers, but it’ll be interesting. In the meantime, I’m loving that I can create beautiful “oil paintings” with words.

_**You seem to have a strong passion for image generation. Could you share with us an image you created using generative AI tools that particularly amazed you?**_

Great timing! At time of writing, Midjourney 6 just came out. In my many hours experimenting, I’ve coaxed it to create some stunning images. I’ve included 4 of my favorites!

![Midjourney Images]({{ sitePrefix }}/assets/img/articles/2024-01-03-Interview-Adam-Mack/mid_favs.webp)


_**How has your experience been working at Monstarlab? Could you share some highlights or key moments that have shaped your journey here?**_

I’ve been here quite a long time in internet years, growing my career from being a very inexperienced graphic designer, to being a solution architect capable of leading international projects. In many ways, the big opportunities of my career have been largely due to chance, where I accidentally find myself in front of a new opportunity to do something important, and I need to adapt and succeed. So that has always been my takeaway, and my secret to success and longevity in my career growth here:

> Always be ready to seize unusual and unexpected opportunities when they arise. It’ll never be what you expected.

_**Reflecting on your early days, what is your earliest memory related to coding? Could you tell us about your first project and the experience of working on it?**_

Growing up, before DOS and Windows machines, we had several Atari personal computers around the house, and one I remember most fondly was the 800. I was fascinated by early games like Time Bandit and Pitfall, and it led to me asking my father questions about how games work. That led us to doing little programming exercises together out of magazines, plugging them into BASIC and hoping they ran. I remember one time he led us in daisy chaining a bunch of hardware together for the intended outcome of capturing still images from a VHS and printing them in high contrast black and white on a dot matrix printer as “coloring book” images. Wizardry like that is what sparked my life long interest in software and hardware development.

_**Is there a book, not necessarily related to programming, that has had a significant impact on your thinking or approach to your work?**_

I always love an opportunity to plug this book - I read it in college, and it changed my whole way of approaching critical thinking: _How We Know What Isn’t So by Thomas Gilovich_. It’s all about the simple lapses in reason that affect everyone, every single day, and how you can recognize and work around them. It really impacted how I evaluate my own thinking and beliefs, and how I approach problem solving.


Behind the screens: Interview with Adam Mack, Lead Product Manager at Monstarlab


C language is never going away. Not only is it portable, efficient and performant, but it's also
very easy to find a compiler for any platform.
[SQLite](https://www.sqlite.org/index.html) might be one of the most well known C library, but it's not
the only one.

- [Lua](https://www.lua.org/) scripting language.
- [curl](https://curl.se/) command-line tool for transferring data specified with URL syntax.
- [SDL](https://github.com/libsdl-org/SDL) cross-platform development library designed to
provide low-level access to audio, keyboard, mouse, joystick, and graphics hardware via OpenGL and Direct3D.

Python libraries, such as [Numpy](https://numpy.org/), also reach out to C in order
to speed up certain parts of the code.

Recently, while working with images, we discovered that UIKit can't decompress all JPEG2K images.
Since this was a mandatory requirement, we had to find a solution.
After a bit of digging, we came across [OpenJPEG](https://www.openjpeg.org/).
We did a few tests and confirmed that it was able to decompress the images that UIKit couldn't.
The only issue remaining was that **OpenJPEG** was a C library.

To use **OpenJPEG** in an iOS SDK, there were two options:

1. statically compile the library and bundle it in an XCFramework
2. wrap the source code in a Swift package

For option 1, we'd have to deal with CMake, compile it for multiple architectures, archive them in Frameworks
and finally package them in an XCFramework bundle. That seemed to be very time-consuming, so we went with option 2,
which is not without its quirks, but pretty straightforward once you understand it. So let's C how to do that, *wink* *wink*.


In this article, we're going to create a small wrapper around the Xoshiro256++ PRNG (Pseudorandom Number Generator).
This is a great PRNG, and it's about 10x to 28x faster than
`UInt64.random`.


If you wanted to implement this algorithm in Swift you could do it like this.


public final class Xoshiro256PlusPlus: RandomNumberGenerator {
    public typealias Seed = (UInt64, UInt64, UInt64, UInt64)
    private var s: Seed

    public init(seed: Seed = Xoshiro256PlusPlus.randomSeed()) {
        self.s = seed
    }

    public static func randomSeed() -> Seed {
        return (
            UInt64.random(in: UInt64.min...UInt64.max),
            UInt64.random(in: UInt64.min...UInt64.max),
            UInt64.random(in: UInt64.min...UInt64.max),
            UInt64.random(in: UInt64.min...UInt64.max)
        )
    }

    /// Generates a pseduo random number.
    public func next() -> UInt64 {
        let result = rotl(s.0 &+ s.3, 23) &+ s.0
        let t = s.1 << 17
        s.2 ^= s.0
        s.3 ^= s.1
        s.1 ^= s.2
        s.0 ^= s.3
        s.2 ^= t
        s.3 = rotl(s.3, 45)

        return result
    }

    private func rotl(_ x: UInt64, _ k: Int) -> UInt64 {
        return (x << k) | (x >> (64 - k))
    }
}
```

This is just a simple example, but it's sufficient to explain how public and private interfaces
work in C and how to structure your Swift package. The C code is still
around 10% faster than the Swift version.

You can get the original Xoshiro code at [Xoshiro Generators](https://prng.di.unimi.it/).


## How to use C in Swift packages

Before anything else let's create the library package.

```bash
$ mkdir Xoshiro && cd Xoshiro
$ swift package init --name "Xoshiro" --type library
$ open Package.swift
```

Add two new **targets**, `CLib` and `CLibTests` and create the
respective folders.

```swift
.target(
    name: "CLib"
),
.testTarget(
    name: "CLibTests",
    dependencies: ["CLib"]
),
```

Then let's add the actual code. By default, all header files added in
the include folder are public.

#### Small helper function for rotating bits left

```c
// Xoshiro/Sources/CLib/include/bitops.h
#ifndef bitops_h
#define bitops_h

#include <stdint.h>

extern uint64_t rotl(const uint64_t x, int k);

#endif /* bitops_h */
```

```c
// Xoshiro/Sources/CLib/bitops.c
#include "bitops.h"

inline uint64_t rotl(const uint64_t x, int k) {
    return (x << k) | (x >> (64 - k));
}
```

#### Xoshiro256++ PRNG

```c
// Xoshiro/Sources/CLib/include/xoshiro256.h
#ifndef xoshiro256_h
#define xoshiro256_h

#include <stdint.h>

void xoshiro256_seed(uint64_t s1, uint64_t s2, uint64_t s3, uint64_t s4);
uint64_t xoshiro256_next(void);

#endif /* xoshiro256_h */
```

```c
// Xoshiro/Sources/CLib/xoshiro256.c
#include "xoshiro256.h"
#include "bitops.h"

static uint64_t s[4];

void xoshiro256_seed(uint64_t s1, uint64_t s2, uint64_t s3, uint64_t s4) {
    s[0] = s1;
    s[1] = s2;
    s[2] = s3;
    s[3] = s4;
}

uint64_t xoshiro256_next(void) {
    const uint64_t result = rotl(s[0] + s[3], 23) + s[0];
    const uint64_t t = s[1] << 17;

    s[2] ^= s[0];
    s[3] ^= s[1];
    s[1] ^= s[2];
    s[0] ^= s[3];
    s[2] ^= t;
    s[3] = rotl(s[3], 45);

    return result;
}
```

We can easily test if this is generating uniform distributions by
creating a simple test that checks when we generate N random
numbers if all possible results share roughly the same percentage.

```swift
import XCTest
@testable import CLib

final class CLibTests: XCTestCase {
    func testXoshiro() throws {
        let s = (1...4).map { _ in UInt64.random(in: UInt64.min...UInt64.max) }

        xoshiro256_seed(s[0], s[1], s[2], s[3])

        let results = NSCountedSet()
        let options: UInt64 = 100
        let samples = 1_000_000

        for _ in 0..<samples {
            results.add(UInt8(xoshiro256_next() % options))
        }

        for option in 0..<options {
            let actual = Double(results.count(for: option)) / Double(samples)
            let expected = 1.0 / Double(options)

            XCTAssertEqual(actual, expected, accuracy: 0.01)
        }
    }
}
```

There's one problem though, we have put everything in the `include`
folder which means every function in both `bitops.h` and `xoshiro256.h`
are public.
The user doesn't need to know about `uint64_t rotl(const uint64_t x, int k)`
in `bitops.h`, so how do we make it private?


One simple way is to move it to the root folder of its target,
like this.


```commandline
Xoshiro
├── Package.swift
└── Sources
    ├── CLib
    │   ├── bitops.c
    │   ├── bitops.h
    │   ├── xoshiro256.c
    │   ├── include
    │   │   └── xoshiro256.h
    └── Xoshiro
        └── Xoshiro.swift
```

This causes a `'bitops.h' file not found` error.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-12-22-C-In-SPM/FileNotFound.webp>
</figure>

To fix this you have to tell SPM where to search for header files
by adding a header search path.

```swift
.target(
    name: "CLib",
    cSettings: [
        .headerSearchPath(".")
    ]
),
```

Personally, I prefer being more explicit and make it clear what's
public and private by creating `private` and `public` folders.

```commandline
Xoshiro
├── Package.swift
└── Sources
    ├── CLib
    │   ├── private
    │   │   ├── bitops.c
    │   │   ├── bitops.h
    │   │   └── xoshiro256.c
    │   └── public
    │       └── xoshiro256.h
    └── Xoshiro
        └── Xoshiro.swift
```

And then explicitly set the public headers' path.

```swift
.target(
    name: "CLib",
    publicHeadersPath: "public",
    cSettings: [
        .headerSearchPath("private")
    ]
),
```

## How to add a Swift interface

Now the easiest part, wrapping the C interface with a Swift interface.

```swift
import CLib

public final class Xoshiro256: RandomNumberGenerator {
    public typealias Seed = (UInt64, UInt64, UInt64, UInt64)

    public init(seed: Seed? = nil) {
        if let seed {
            xoshiro256_seed(seed.0, seed.1, seed.2, seed.3)
        } else {
            xoshiro256_seed(
                UInt64.random(in: UInt64.min...UInt64.max),
                UInt64.random(in: UInt64.min...UInt64.max),
                UInt64.random(in: UInt64.min...UInt64.max),
                UInt64.random(in: UInt64.min...UInt64.max)
            )
        }
    }

    public func next() -> UInt64 {
        return xoshiro256_next()
    }
}
```

# Conclusion

Using C in Swift packages is pretty straightforward once you
know how to structure your code. Being able to use C is especially
useful when dealing with low-level code that needs to be performant.
Unsafe Swift syntax/API is also a bit too verbose for my liking, so I
think it's great that I get to call out to C whenever needed.

I hope this tutorial has made integrating C in Swift packages less
daunting.

### Related Articles
- [Apple - C Interoperability](https://developer.apple.com/documentation/swift/c-interoperability)
- [Apple - Unsafe Swift](https://developer.apple.com/videos/play/wwdc2020/10648/)
- [Apple - Safely manage pointers in Swift](https://developer.apple.com/videos/play/wwdc2020/10167/)


_Article Photo by [Iñaki del Olmo](https://unsplash.com/@inakihxz)_


How to use C in Swift Packages


In the [previous article]({{ baseurl }}/post/2023/01/02/How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS), we discussed how to set up a **"GitHub hosted runner"** with WordPress. Now, in this part, we will explore how to achieve the same deployment process using a **"Self-hosted runner"**

## GitHub Branches and Web Servers Architecture Recap:

Before diving into the self-hosted runner setup, let's quickly recap our GitHub branches and web servers architecture. We have multiple branches, including **"dev-deploy"** and **"prod-deploy."** The **"dev-deploy"** branch has a dedicated GitHub runner for the development server. After successful testing on the dev server, we are ready to deploy changes to the production server using the **"prod-deploy"** branch.

![Github Branches]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/github.webp

| Branch Name       | Our Actions Based on it                                                                    |
| ----------------- | ------------------------------------------------------------------------------------------ |
| **_master_**      | Here we keep all up-to-date codebase.                                                      |
| **_prod-deploy_** | If any code is merge in this branch, We run a GitHub Action for the production deployment. |
| **_dev_**         | We work on this branch for fixing issues and building features.                            |
| **_dev-deploy_**  | If any code is merge in this branch, We run a GitHub Action for development deployment.    |

<br />
In our production environment, we have an added layer of security provided by a VPN. This VPN connection is necessary to access the webserver, and it is the primary reason why we opted for a self-hosted runner this time.

![server architecture]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/aws.webp

## Options for Self-Hosted Runners:

With self-hosted runners, we have three options to consider:

1. **Repository-level:** Runners dedicated to a single repository.
2. **Organization-level:** Runners capable of processing jobs for multiple repositories within an organization.
3. **Enterprise-level:** Runners assigned to multiple organizations in an enterprise account.
   Our example will be based on the repository-level, but you can choose the most suitable option based on your requirements.

## Setting Up the Self-Hosted Runner:

Navigate to the repository settings page and click on **"Actions,"** then select **"Self-Hosted Runner."**
![Github Settings]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/github-settings.webp

Choose the appropriate **"Runner image**" available on your server and select the corresponding **"Architecture"** (e.g., "Linux" and "x64").
![select image]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/image-select.webp

The installation and configuration of the self-hosted runner involve three steps:

1. Download
2. Configure
3. Using your self-hosted runner.

![self hosted runner]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/Self-Hosted-Runner.webp

##### Download:

In this step, we need to log in to our own server and execute given commands.

##### Configure:

During the configuration, you may encounter some prompts. For the **"group name,"** you can use the default one. Give a name to the runner, such as **"production-deploy,"** which will appear in your runner page on the Github. Finally, set an label, for example, **"prod-deploy."**
![github action runner configure]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/configure.webp

##### Using your Self-Hosted Runner:

Now, you can start listening for job requests and to ensure continuous listening in the background by typing

```sh
sudo ./svc.sh install
sudo ./svc.sh start
```

With the self-hosted runner successfully configured and listening, any code changes merged into the **"prod-deploy"** branch will trigger the runner, facilitating deployment to our production server.
![github action configure]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/configured-successfull.webp

## Creating the Deployment Automation YML File:

###### Step 1:

Create a folder named **".github"** in the root of project directory. Inside the **".github"** directory, create another folder named **"workflows."** Within this **"workflows"** folder, create a file named **"prod_deploy.yml."** The file extension must be **".yml."** All the deployment automation code will be written in this file.

###### Step 2:

Provide a meaningful "name" for the workflow. This will help in identifying the workflow in the GitHub Actions interface.

![github action name]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/action-name.webp

###### Step 3:

Configure the **"on"** event to specify when the GitHub Action should run. For instance, we want the workflow to trigger when code is merged into the **"prod-deploy"** branch.

![github action name]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/action-run.webp

###### Step 4:

Set a default directory before running the **"jobs"** Since our root directory hasn't the WordPress theme, we can set the theme directory to avoid changing directories in each step.

![github action name]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/working-directory.webp

###### Step 5:

Define the **"jobs"** within the GitHub workflow. Each job contains a series of **"steps,"** which can be actions or shell scripts. Here, we create a job and specify that it should run only if code is merged into the **"prod-deploy"** branch. We also add the **"prod-deploy"** label, which ensures that the self-hosted runner runs on the associated server.

Now let's see what it looks like when we write all the code together.

<script src="https://gist.github.com/Rasel-Mahmud/bdbe8200c07121a59870379b524e5844.js"></script>

##### Let's break down each step in the deployment process:

- **`checkout:`** This step involves checking out our files into the virtual server using the GitHub checkout action from https://github.com/actions/checkout. This ensures that we have the latest code available for the deployment process.

- **`cache:`** The cache step is used to store cached data, allowing the theme build process to be faster after the first execution, provided the cache exists. For more information on the cache action, you can refer to https://github.com/actions/cache.

- **`Install Node dependency:`** In this step, we run an action to install the npm package dependencies required for our project. This ensures that the necessary Node modules are available for the subsequent build process.

- **`Install PHP dependency:`** Here, we execute an action to install the PHP package dependencies that our WordPress theme relies on. This ensures that the required PHP libraries and components are available for the theme build.

- **`Build The Theme:`** This step involves running an internal package command to build our WordPress theme. For this purpose, we are using the ["Tonic starter"](https://labs.tonik.pl/theme/) theme, which provides a solid foundation for our WordPress Theme development.

- **`Move Theme:`** After the theme is successfully built, we need to move it to the appropriate location on our server. This step utilizes basic shell commands to carry out the process. For example, it may involve removing the existing theme using and moving the current build folder to the appropriate location using

```sh
sudo rm -rf /theme-location/theme-name/
sudo mv * /theme-location/theme-name/
```

These commands should not be exposed publicly as they reveal sensitive server file paths. To keep these details secure, we can follow the steps below:

1. Go to the Repository and click on the **"Settings"** tab
   ![github Settings]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/setting-one.webp
2. In the left side: menu, you will find **`"actions"`** under the **`"secrets"`** menu.
   ![github setting]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/setting-two.webp
3. Click on **`"New repository secret"`**.
4. Provide a **`"name"`** on that specific **`"secret"`** which we can access it by its name **_`${{ secrets.THEME_LOCATION }}`_** here **`"THEME_LOCATION"`** was the name. <br />

![github setting]({{ sitePrefix }}/assets/img/articles/2023-12-14-How-to-setup-CI-CD-pipeline-for-WordPress-with-GitHub-Actions-and-AWS-part-2/theme-location.webp

With the self-hosted runner and GitHub workflow configured, we have established an automated deployment process for our WordPress theme. The self-hosted runner listens for changes in the "prod-deploy" branch, allowing us to deploy the latest changes to our production server. This setup ensures smoother and more efficient deployment while maintaining an extra layer of security through the VPN connection.

## Resources

_Article Photo by [Praveen Thirumurugan](https://unsplash.com/@praveentcom)_


How to setup CI/CD pipeline for WordPress with GitHub Actions and AWS (Part 2)

 

Welcome, fellow developers! Here's a deep dive into the core concepts of JavaScript and how TypeScript could enhance them. Regardless of the level of knowledge, mastering the foundations of JavaScript is essential. We want to delve deep into the core concepts of JavaScript and understand how TypeScript changes them. A superset of JavaScript, TypeScript delivers substantial enhancements that give us a full view of the language's capabilities.

This journey's goal is to teach you a thorough understanding of JavaScript's underpinnings as well as the significant enhancements made by TypeScript. By the time you complete reading this article, you will have the knowledge required to apply both in your development projects. Now let's start this illuminating voyage.

# **Basic principles of JavaScript**

JavaScript was first primarily used as a simple scripting language to improve web interactivity. However, it lacked a number of sophisticated capabilities found in contemporary programming languages. JavaScript has now evolved into a potent tool for front-end and back-end development with the addition of TypeScript.

Static typing, interfaces, enums, and generics are among the fundamental concepts introduced by TypeScript, a statically typed superset of JavaScript.

It is now an essential component of contemporary online and application development and supports functional, object-oriented, and asynchronous programming paradigms.

There are three main parts to a core JavaScript engine:
- Call stack 
- Memory/variable environment 
- Thread of execution

We will address each of these elements in turn in the following sections, offering in-depth analyses for a thorough grasp of how TypeScript improves JavaScript development.

### Unlocking the Power of Reusable Code: Exploring JavaScript Functions with TypeScript

Within the realm of programming, functions perform as standalone programs when utilised, particularly in JavaScript (which is now further enhanced by TypeScript), and also code becomes more robust and dependable with greater type safety.

Two key considerations need to be made in order for this to function in TypeScript:

- **Thread of Execution:** This idea entails going through the TypeScript code for the function line by line, much like when you follow a recipe to the letter.

- **Memory:** This memory space enables the function to handle and store crucial information or data needed for its activities, much like a defined storage area. Type annotations in TypeScript facilitate the definition and administration of data structures, improving memory management.

**Execution context** is the term used in TypeScript to describe the confluence of these two fundamental components. To ensure that the function does its assigned tasks precisely and type-safely, this framework serves as its TypeScript operating environment.

### A Detailed Overview of JavaScript and TypeScript Function Definition and Usage

JavaScript and its powerful extension TypeScript both employ the 'function' keyword and the function name to build functions. The syntax for the short arrow function (=>) is an alternative.

Just remember that in order to use a function in TypeScript—that is, to call, execute, or run it—it must always be enclosed in parenthesis. Code reliability and maintainability are enhanced when appropriate types of arguments are passed, thanks to TypeScript's type system. With TypeScript, you may define and use functions with more confidence that your code is valid.

### Tracking Functions in the JavaScript CallStack

The "global" main function in JavaScript is where all code runs. To comprehend how JavaScript manages this global environment during the execution of a function, consider the "Call Stack".

The stack of plates that represents the call stack is made up of individual function calls. As an illustration:

```javascript
function greet(name) {
  console.log(`Hello, ${name}!`);
}

function sayHello() {
  greet("Alice");
  greet("Bob");
}

sayHello();
```

- The execution of `sayHello()` adds to the call stack. Two calls performed inside of `sayHello()` to `greet()` have also been added to the Call Stack. The Call Stack tracks the order in which functions are executed.
- Handling Nested Functions: The Call Stack efficiently handles nested function management. Every function is removed from the stack after it is finished. For example:
  - `sayHello()` is now included in the Call Stack.
  - `sayHello()` now includes `greet("Alice")`.
  - `greet("Alice")` terminates and is removed.
  - An additional `greet("Bob")` is attached.
  - `greet("Bob")` terminates and is removed.
  - Upon completion of `sayHello()`, the Call Stack is cleaned.

The Call Stack is essential for debugging and code optimisation since it ensures that functions in JavaScript's global context are executed in a logical order.

# **Comprehending Higher Order Functions & the JavaScript Callback**

Passing a function as an argument to another function can look strange, especially when execution happens later. However, **"callback functions."** is the foundation for asynchronous programming with TypeScript and JavaScript.

Callback functions, or functions that are passed as arguments, are crucial for managing asynchronous tasks in JavaScript and TypeScript. For example, they specify what should happen after an API request is processed or a user activity is finished.

These days, higher-order functions like map, filter, and reduce are part of the toolkit in TypeScript and JavaScript. Also, Promises and the succinct async/await syntax have strong type support in TypeScript.

### Demystifying JavaScript's Functional Programming: Unleashing Pro-Level Functions and Technical Interview Success.

Using functions, which makes it possible to use strong pro-level functions like reduce, filter, and map, is one of the most misunderstood topics in JavaScript. Functions are important to functional programming. A function is the foundation of a professional mid- to senior-level engineering interview and the means by which our code becomes more declarative and legible in Codesmith. 

Using the following syntax, one could write a function that returns the square of 10:

```javascript
function tenSquared() {
  return 10*10;
}

tenSquared() // 100
```

```javascript
function nineSquared() {
  return 9*9;
}

nineSquared() // 81 😞

// And an 8 squared function? 125 squared?
// What principle are we breaking? DRY (Don't Repeat Yourself)
```

```javascript
// We can generalize the function to make it reusable
function squareNum(num) {
  return num*num;
}

squareNum(10); // 100
squareNum(9);  // 81
squareNum(8);  // 64
```

### Generalizing Functions

Code flexibility in JavaScript and TypeScript is largely dependent on the idea of generalising functions. "Parameters" (placeholders) allow us to perform our functionality on any data without having to decide on it beforehand.

When we run the function, supply a real value ('argument').

This same approach applies to higher-order functions.

- Until we run our function, we might not want to decide exactly what part of our functionality is.

### Understanding Higher-Order Functions in JavaScript & TypeScript

Higher-order functions are essential to TypeScript and functional programming alike. They take functions as parameters and return functions as outcomes, which improves the flexibility and reusability of the code.
Code is kept DRY (Don't Repeat Yourself) by using these utilities to simplify it. They make legible code possible through reduce, filter, and map operations. Add to this combination type safety with TypeScript.

Knowing higher-order functions is essential for professional interviews. Additionally, they are essential to asynchronous JavaScript, supporting ideas like async/await and Promises.

Whether for interviews or contemporary online programmes, knowing higher-order functions is essential to becoming proficient in JavaScript and TypeScript.

Understanding JavaScript's Arrow Functions:

```javascript
// Introducing arrow functions - a shorthand way to save functions

function multiplyBy2(input) { return input * 2; }

const multiplyBy2 = (input) => { return input*2 }

const multiplyBy2 = (input) => input*2

const multiplyBy2 = input => input*2

const output = multiplyBy2(3) // 6
```

`createMultiplier`, a higher-order function, uses arrow syntax to return another function. This function multiplies a number by a predetermined amount. One specific example of a `createMultiplier` that doubles any input integer is `multiplyBy2`. By utilising `multiplyBy2(3)`, it is evident that arrow functions can be employed to generate specialised functions that exhibit elegant behaviour, such as returning 6.

# **Understanding JavaScript Closures (Scope and Execution Context)**

A closure in JavaScript is a function that works even after the outer function has finished running, allowing it to access variables from its outer scope.

In JavaScript, a closure is a function that returns from the execution of the outer function with access to variables from its outer scope.

Closures are highly helpful even though they were once regarded to be tough. They are now commonly understood. In order to construct reusable modules and improve code structure, they are usually required to maintain private and well-organized data. The usage and understanding of closures in modern JavaScript has increased.

Closures perform Memory-Related tasks

- We establish a live store of data (local memory/variable environment/state) for the function's execution context when our functions are invoked.
- After the function has finished running, its local memory is removed (apart from the returned value).

But what if our functions could save real-time data between calls?

This would allow our function definitions to be linked to a cache or permanent memory.

But it all starts with us returning a function from another function.

The Connection:

Upon definition, a function is bound to the Local Memory (sometimes called the "Variable Environment") in which it is defined.

The "Backpack":

```javascript
function outer() {
    let counter = 0;
    function incrementCounter() {
        counter++;
    }
    return incrementCounter;
}

const myNewFunction = outer();
myNewFunction();
myNewFunction();

const anotherFunction = outer();
anotherFunction();
anotherFunction();
```

We obtain `incrementCounter's` function definition from `outer` and give it a new name, `myNewFunction`.

- The link to `outer's` live local memory is maintained; it is 'returned out' and attached to `incrementCounter's` function definition.
- As a result, the function `outer's` local memory is still linked to `myNewFunction` even if its execution context has long ago terminated.
- Running `myNewFunction` in global mode causes it to look first in its own local memory and then in its `"backpack"`, as would be expected.

Thanks to closure, our functions now have enduring memory and an entire new toolkit for writing well-written code.

# **Understanding Asynchronous and Event Loop in JavaScript and TypeScript**

Using TypeScript and JavaScript for asynchronous programming is necessary to create applications that are responsive and efficient. Asynchronous operations enable activities to be completed concurrently without interfering with the main thread of processing, in contrast to synchronous operations that run code one after the other.

An essential part of TypeScript and JavaScript, the event loop makes sure that asynchronous operations are carried out without blocking, resulting in responsive behaviour and economical use of resources.

Promises, the async/await syntax, and the event loop simplify asynchronous tasks in modern JavaScript and TypeScript. This improves the readability and maintainability of the code while guaranteeing smooth performance.

**Important elements in this procedure consist of:**

One essential ES6 feature that makes writing asynchronous programming easier is promises.
- Asynchronicity: The ability to handle concurrent tasks is what enables dynamic web applications.
- Event Loop: The asynchronous task management triage mechanism built into JavaScript.
Web browser APIs, microtask queues, and callback queues are crucial parts of asynchronous process management.

The three primary components of the JavaScript engine are as follows:
- Thread of execution 
- Memory and variable environment
- Call stack

Beyond the capabilities of ECMAScript 5 (ES5), advancements such as Promises and a more efficient Event Loop have been included in more recent iterations of JavaScript and TypeScript. In comparison to the ES5 era, these improvements increase the efficiency and maintainability of handling asynchronous operations and apply to both Node.js and web browsers.

**ES6+ Solution with Promises**

Promises provide a two-pronged approach to asynchronous task handling in modern JavaScript and TypeScript.
They start web browser work in the background.
- Without delay, they provide a placeholder object, or promise.

These Promises adhere to the following guidelines for running code that is deferred asynchronously:
A microtask queue is used to store promise-deferred functions, and a task queue (called the Callback queue) is used for callback functions once the associated Web Browser Feature (API) has finished.
Functions are added to the Call stack (that is, executed) once all global code has run and the Call stack is empty. This state is kept an eye on by the Event Loop.

Promises let JavaScript and TypeScript do the following things when used with Web APIs, Callback & Microtask Queues, and the Event Loop:
- Develop non-blocking programmes that enable the execution of multiple threads of code concurrently without waiting.
- Take care of scheduling the completion of Browser features automatically, guaranteeing effective code execution.
Act as the foundation for contemporary online applications, making it possible to create web solutions quickly and without stuttering.

# **Classes & Prototypes (OOP)**

For object-oriented programming, JavaScript has historically employed prototypes, which may be confusing to developers used to working with class-based systems. 

However, object-oriented programming in JavaScript became more recognisable to developers of other languages with the release of ECMAScript 6 (ES6), which introduced a class syntax. Although JavaScript classes still use prototypes internally, the new syntax offers a more organised and understandable method for generating and extending objects.

Important things to remember:

- Classes are a commonly used paradigm for structuring complex code in TypeScript and JavaScript.
- Prototype chain, an efficient tool in and of itself that enables the replication of Object-Oriented Programming (OOP), is one such background-working component.
- The difference between __prototype__ and __proto__ must be understood for effective OOP in JavaScript and TypeScript.
- Programming is made easier by using the `new` and `class` keywords, which automate the generation of objects and methods.

# __proto__:

Every JavaScript object, including arrays, functions, and other objects, has a property called __proto__. JavaScript initially verifies that a property or function on an object actually exists before allowing you to access it.

# __prototype__:

In TypeScript and JavaScript, every function has the property prototype. The prototype property of a constructor function becomes the prototype of the newly generated object when it is defined and instances are created using the new keyword. 

A Guide for Writing and Using Code
1. Save data (for example, user 1 and user 2's quiz scores).
2. Use the data to run code (functions) (e.g., enhance user 2's score)

Simple! Then why is development challenging? I have to save a lot of users in a quiz game, along with administrators, quiz questions, quiz results, league tables, and other data and functionality.

Where's the functionality when I need it in 100,000 lines of code?
- How can I ensure that the functionality is only applied to appropriate data?

To make things simple, let's say I save each user's specific data in my app:

| User  | Name       | Score |
|-------|------------|-------|
| user1 | Tim        | 3     |
| user2 | Stephanie  | 5     |

<br>
And (simplifying again!) the features I have for every user are: greater functionality (there would actually be a multitude of features!).

How might my entire set of features and information be in one place?

```javascript
// Solution 1. Generate objects using a function

function userCreator(name, score) {
    const newUser = {};
    newUser.name = name;
    newUser.score = score;
    newUser.increment = function() {
        newUser.score++;
    };
    return newUser;
}

const user1 = userCreator("Will", 3);
const user2 = userCreator("Tim", 5);
user1.increment();
```
**Problems**: Our computer has to allocate memory for all of our data and features each time we add a new user. But all we are doing is copying functions. Does a better approach exist?

**Advantages**: It's easy to understand and consider!

```javascript
// Solution 2: Using the prototype chain

function userCreator(name, score) {
    const newUser = Object.create(userFunctionStore);
    newUser.name = name;
    newUser.score = score;
    return newUser;
}

const userFunctionStore = {
    increment: function() { this.score++; },
    login: function() { console.log("Logged in"); }
};

const user1 = userCreator("Will", 3);
const user2 = userCreator("Tim", 5);
user1.increment();
```

## Understanding TypeScript to Enhance JavaScript Development

We studied the hard JavaScript sections as well as the basic concepts and challenges of this dynamic language. All things considered, the rise in popularity of TypeScript, is one of the most significant advancements in JavaScript lately.

Adding TypeScript to a project has several advantages:

1. Maintainability of Code
2. Enhanced Developer Productivity
3. Improved Collaboration
4. Scalability
5. Reduced Number of Bugs
6. Ecosystem and Community

Furthermore, TypeScript facilitates faster code reviews, more comprehensible code, and easier onboarding of new team members—all of which enhance developer collaboration.

# Conclusion

As a result of our thorough investigation of JavaScript and its potent extension, TypeScript, we have gained a profound understanding of the underlying ideas that underpin this adaptable programming language. We have studied the management of intricate processes by JavaScript, thanks to its asynchronous features such as callbacks and higher-order functions. The event loop and closures, which are crucial to comprehending JavaScript's execution mechanism, have also been clarified.

We are more equipped to take on challenging coding tasks as we continue to explore the nuances of JavaScript, enhanced by TypeScript's extensions. Our objective is to develop applications that are resilient, adaptable, and scalable to the demands of the ever-changing digital landscape. So, fellow coders, let's embrace TypeScript and JavaScript to the fullest and use their combined strength to innovate and change the digital landscape.

Happy Coding!



_Article Photo by [Unsplash](https://unsplash.com)_

Mastering Modern JavaScript: From Core Concepts to Advanced Techniques

 

AndroidX Bluetooth is a new addition to the [Jetpack Suite](https://developer.android.com/jetpack) of libraries. While currently in its alpha stage, the system already provides robust safety measures addressing [common pitfalls](https://punchthrough.com/android-ble-development-tips/) in Android BLE development. Moreover, it establishes a clear trajectory for further improvements. Official Android Developer Documentation describes AndroidX Bluetooth as:

> This is the initial release of AndroidX Bluetooth APIs that provides a Kotlin API surface covering Bluetooth LE scanning and advertising, and GATT client and server use cases. It provides a minimal API surface, clear thread model with async and sync operations, and ensures all methods be executed and provides the results.

Sounds very promising, but less talk, more action, let's dive deeper into what AndroidX Bluetooth library brings to the table!

## How to use AndroidX Bluetooth library

First of all, import the artifact in your `build.gradle`.

```groovy
implementation "androidx.bluetooth:bluetooth:1.0.0-alpha01"
```

*Note: current minSdk of the artifact might be too high (33 at the point of writing this article) and it is intended. They are gradually lowering the minSdk as part of integration testing/for people to not use it in production since it is still in alpha.*

## Working with the library

With the basic introductions done, it is time to buckle up and start coding! For the sake of simplifying the article we will skip the permission handling, checking whether device has bluetooth and if it is enabled and move straight to the specifics of the AndroidX Bluetooth library.

### `BluetoothLe` class

`BluetoothLe` class is the main entry point of the library. Internally, it contains `GattClient` and `GattServer` classes which are responsible for the GATT client and GATT server role, respectively. Every usecase you have will go through this class.

Currently `BluetoothLe` class has 4 main methods:
 - `scan` - launches peripheral scanner
 - `connectGatt` - handles the connection to the peripheral
 - `advertise` - starts BLE advertiser
 - `openGattServer` - starts GATT server

To instantiate `BluetoothLe` class, simply call its constructor where you have to supply it with `Context`.

## GATT Client/Central

Now that we have our `BluetoothLe` class set up, let's dive deeper into GATT client/central role and its main responsibilities. That includes scanning for peripherals, connecting and communicating with them via read/write operations.

### Scanning

Let's start with the first operation we have to do when doing any business with the bluetooth devices - the scan.

To launch a scan, simply call the `bluetoothLe.scan(List<ScanFilter>)` method. It has one parameter, which is a list of scan filters and it returns a `Flow` of `ScanResult`s, which is very convenient! No more callback hells, instead we have a **cold** flow that emits devices once they are detected by the hardware scanner (and passes your scan filters).

You can notice that there is no way to set  `ScanSettings` for the scan since it doesn't exist as a parameter for the function. `ScanSettings` allows developers to set the scan mode, match mode, PHY, etc. Since under the hood it still uses framework's `bluetoothLeScanner`, it is still mandatory to specify, but for now it is not implemented in AndroidX library and it uses default `ScanSettings` instead:

```kotlin
val scanSettings = ScanSettings.Builder().build()  
bleScanner?.startScan(fwkFilters, scanSettings, callback)
```

There is also no method to stop the scan, but by checking the code inside the library we can see the following code:

```kotlin
fun scan(filters: List<ScanFilter> = emptyList()): Flow<ScanResult> = callbackFlow {

  // scan method body
  
  awaitClose {  
    bleScanner?.stopScan(callback)  
  }  
}
```

Notice the `awaitClose`? It suspends the function/coroutine until it is either closed or cancelled and invokes the code block. Therefore in order to stop the scan we would need to cancel the `Job` that collects the scan results. A draft of the code would look like this:

```kotlin
private var scanJob: Job? = null

fun startScanning() {  
  scanJob = viewModelScope.launch {
    try {
      bluetoothLe.scan().collect { scanResult ->
        // do something with the scan results
      } 
    } catch (exception: Exception) {
      // handle scan errors
    }
  }
}  
  
fun stopScanning() {  
  scanJob?.cancel()  
  scanJob = null  
}
```

Looks quite tidy and clean! Now you can take your device from the scan results and connect to it. 

### Connecting

To connect to the peripheral, one should make use of the `connectGatt` method. 

```kotlin
bluetoothLe.connectGatt(BluetoothDevice, suspend GattClientScope.() -> R)`
```

There are two parameters for developers to fiddle with:

 - `BluetoothDevice` - library's wrapper around framework's `BluetoothDevice`. You can receive it from the `ScanResult` when scanning for peripherals (more details in the scanning section above)
 - `suspend GattClientScope.() -> R` - lambda which is marked as suspend. It invokes once the connection with the peripheral is established. Since it is also an extension of `GattClientScope` you can call various methods on it to communicate with the remote device.

Therefore, to connect to the peripheral, you'd need to have something like the following:

```kotlin
// every communication is going through this object once the device is connected  
private var gattClient: BluetoothLe.GattClientScope? = null

try {
  bluetoothLe.connectGatt(bluetoothDevice) {  
    gattClient = this
  }
} catch (exception: Exception) {
  // handle connection errors
}
```

Since lambda is an extension of `GattClientScope`, we can save it to the outside variable, similar to how you'd save `BluetoothGatt` object when working with the framework's BLE SDK.

One thing to keep in mind is that **the connection will be dropped the moment** `suspend GattClientScope.() -> R` **lambda finishes**. To negate that, you need to **suspend** the lambda at the end by calling `awaitCancellation()`. That will also allow you to control when to disconnect from the device, similarly to how we stopped the scan earlier. Therefore, the connectivity block might look like this:

```kotlin
// every communication is going through this object once the device is connected  
private var gattClient: BluetoothLe.GattClientScope? = null
private var connectJob: Job? = null

fun connect() {
  connectJob = coroutineScope.launch {
    try {
      bluetoothLe.connectGatt(bluetoothDevice) {  
        gattClient = this
        awaitCancellation()
      }
    } catch (exception: Exception) {
      // handle connection errors
    }
  }
}

fun disconnect() {
  connectJob?.cancel()  
  connectJob = null  
}
```

After it establishes the connection, the library automatically does the initial flow of setting up the GATT. The flow looks like this: *connection success >> request MTU >> 515 >> discover services >> we're ready!* The only flaw in this is that if the MTU request fails, then the whole connection is dropped. Granted that the MTU request is not a mandatory operation, that seems quite extreme (at least I wasn't been able to connect to my virtual peripheral, because it was failing the MTU request and had to find another way to host a GATT server)

Other than that, the whole connectivity process was smooth and I was able to have a working connectable device in no time.

### Communicating

Once the connection is established, it is time to do some communication, i.e. read/write. Previously we saved `GattClientScope` to an outer variable. This class has a minimum amount of capabilities to start communicating with the remote device. For example, you can check all the characteristics of a certain service by calling the following method:

```kotlin
gattClient.services.first().characteristics
```

Or when you need to read from characteristic or write to it, you can do it all by using the same `GattClientScope`:

```kotlin
// Find the characteristic
// Please note, that this is just an example, you would need to find characteristic
// by its' UUID
val myCharacteristic = gattClient.services.first().characteristics.first()

// read characteristic; this is a suspend function
val result = gattClient.readCharacteristic(myCharacteristic)

// write characteristic; this is a suspend function
gattClient.writeCharacteristic(characteristic, valueToWrite.toByteArray())
```

Besides those standard operations, `GattClientScope` also able to subscribe to characteristic and return flow of values:
```kotlin
// not a suspend function; returns Flow instead
val byteArrayFlow = gattClient.subscribeToCharacteristic(myCharacteristic)
coroutineScope.launch {
  byteArrayFlow.collect { byteArray ->
    // process byte arrays coming from peripheral's notifications 
  }
}
```

### Operation queuing

Developers who worked with Android BLE SDK know that it is forbidden to perform operations in rapid-fire succession. Essentially, if you are not using some third-party library, you would need to implement a queuing mechanism that would execute your BLE operations sequentially, one-by-one (any operation, such as discover services, request MTU, read/write, etc). If you don't do that, chances are that some of the operations will be simply dropped/ignored causing software misbehavior.

In AndroidX Bluetooth that seems to be finally taken care of and developers can finally not worry about creating a queuing mechanism for their operations. That is due to the fact that internally the library uses Kotlin's `Mutex`.  [Mutex](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/) essentially suspends the function until the shared resource is free. But the important part there is that the created mutex is fair: lock is granted in first come, first served order. In another words, Mutex allows us to create a queue for BLE operations. The code snippet below was taken from the AndroidX Bluetooth source code, modified for article's simplicity and enriched with some comments explaining how it works.

```kotlin
val gattScope = object : BluetoothLe.GattClientScope {

  // create Mutex
  val taskMutex = Mutex()
  
  // helper function to make use of Mutex
  // will be used down below in readCharacteristic function
  suspend fun <R> runTask(block: suspend () -> R): R {  
    taskMutex.withLock {  
      return block()  
    }  
  }  
  
  override suspend fun readCharacteristic(characteristic: GattCharacteristic): Result<ByteArray> {  
    // the runTask function locks the mutex  
    return runTask {
      // read the characteristic safely, inside the mutex
      fwkAdapter.readCharacteristic(characteristic.fwkCharacteristic)  
      // the function below suspends until we get the result from the read operation 
      val res = takeMatchingResult<GattClient.CallbackResult.OnCharacteristicRead>(  
        callbackResultsFlow  
      ) {  
        it.characteristic == characteristic  
      }  
  
      // return the result, unlock the mutex
    }  
  }  
}
```

Mutex is used in read, write and subscribe operations, which means that if you somehow rapid-fire read operations along with discover services, request MTU or some other, then you still can theoretically lose some requests, but we are going into an edge case now since AndroidX Bluetooth doesn't currently have the capability to enqueue discover services request through their API.

## GATT Server/Peripheral

Besides being the client most of the time, Android can also be a server! Since Android 5+ some of the devices can act as peripheral/server, but be aware that not all of the devices support that on the hardware level. You can find what phones support that in [that list](https://altbeacon.github.io/android-beacon-library/beacon-transmitter-devices.html).

To open the GATT server, you would need to call `openGattServer` method.

```kotlin
bluetoothLe.openGattServer(List<GattService>, suspend GattServerConnectScope.() -> R)
```

There are multiple design similarities with the previously described operations. Let's dig deeper! There are two parameters, similarly to how client connection works:
 - `List<GattService>` - services and characteristics that the server will hold.
 - `suspend GattServerConnectScope.() -> R` - lambda which is marked as suspend. It executes once the server is opened. Again, similarly to how client connection works, it's also an extension of `GattServerConnectScope`.

`GattServerConnectScope` doesn't have much to offer from the first glance. You can update services by invoking the corresponding method `updateServices(List<GattService>)` and listen to the connect requests by utilizing `connectRequests` Flow. The latter one is a little bit more interesting since you are collecting `GattServerConnectRequest`s. It contains the `BluetoothDevice` from which the request is coming from and methods to accept (has yet another lambda) or reject the request. If you accept the request, you are basically establishing the connection with the client and can start processing read/write requests, notify the client via BLE Notifications, etc.

Therefore, to open the GATT server, you'd need to have something like the following:

```kotlin
bluetoothLe.openGattServer(gattServerServices) { // opens GATT server
   connectRequests.collect { connectRequest ->   // collets clients' connection requests
     connectRequest.accept {                     // accepts the connection request (you can also reject it)
       requests.collect { serverRequest ->       // collects client's read/write requests, etc
         // do something with server request
       }
     }
   }
}
```

You can further improve the snippet above by adding `CoroutineScope.launch` before opening GATT server like we did in the scanning/connectivity before, cancelling it whenever you need to close the server and/or adding parallelism by launching a separate coroutine for each `connectRequest`, etc. The sky is the limit!

## Advertiser

Lastly, you can also have a device advertiser. You can do it by calling the `advertise` method. 

```kotlin
bluetoothLe.advertise(AdvertiseParams, (suspend (@AdvertiseResult Int) -> Unit)?)
```

This method has two parameters:

 - `AdvertiseParams` - provides a way to adjust advertising preferences and advertise data packet.
 - `(suspend (@AdvertiseResult Int) -> Unit)?` - an optional block of code that is invoked when advertising is started or failed.

`AdvertiseParams` has a lot of preferences to modify, let's go over them one-by-one:

- `shouldIncludeDeviceAddress` - sets whether the device address will be included in the advertisement packet.
- `shouldIncludeDeviceName` - sets whether the device name will be included in the advertisement packet.
- `isConnectable` - sets whether the advertisement will indicate connectable.
- `isDiscoverable` - sets whether the advertisement will be discoverable. Note that it would be ignored under API level 34 and `isConnectable` would be used instead.
- `durationMillis` - sets advertising duration in milliseconds. Supports values from 0 to 655350.
- `manufacturerData` - sets a map of company identifiers to manufacturer specific data.
- `serviceData` - sets a map of 16-bit UUIDs of the services to corresponding additional service data.
- `serviceUuids` - sets a list of service UUIDs to advertise.

As you can see, there are a lot of preferences to fiddle with, but more importantly, under the hood, it has all the conditionals to ensure backwards compatibility so you don't need to have a hell of API-level version checks. It is all already done! And that applies to all of the operations we have gone through in this article such as scanning, connectivity, hosting a GATT server, etc!

Lastly, this is how the code would look like if you need to advertise your device:

```kotlin
val advertiseParams = AdvertiseParams(  
  shouldIncludeDeviceAddress = false,  
  shouldIncludeDeviceName = false,  
  isConnectable = false,  
  isDiscoverable = false,  
  durationMillis = 0,  
  manufacturerData = mutableMapOf(),  
  serviceData = mutableMapOf(),  
  serviceUuids = mutableListOf(),  
)
bluetoothLe.advertise(advertiseParams) {
  // do something with the AdvertiseResult
}
``` 

## Room for Improvement

There is definitely room for improvement for the library. Some of them include:

- Consistency. Some methods of `BluetoothLe` suspend the function and you need to cancel it to interrupt the operation and other methods don't suspend the function and you have to add `awaitCancellation` yourself (looking at you `connectGatt` method 👀).
- Make request MTU not mandatory when establishing the connection with the peripheral.
- Allow developers to modify `ScanSettings` when scanning for peripherals, but I am sure it will come in future versions.

## Conclusion

To wrap it up, here are the key takeaways for the AndroidX Bluetooth library:

 - AndroidX Bluetooth is a Jetpack library that adds synchronous way to handle bluetooth operations.
 - AndroidX Bluetooth allows developers to scan, connect, communicate, create their own GATT servers and advertise their devices.
 - AndroidX Bluetooth is heavily leveraging Kotlin Coroutines, which makes collecting results and/or cancelling operations relatively easy.
 - AndroidX Bluetooth handles all API level version specifics for you. Less version checks conditionals = good.
 - It is still in alpha and a lot of features are missing and there is room for improvement, but it is a very good start and makes third-party Android BLE libraries a bit less mandatory for the real world development


_ Article Photo by [Sten Ritterfeld](https://unsplash.com/@stenslens) on [Unsplash](https://unsplash.com/photos/black-and-white-remote-control-psKil0FkS58)


First look at AndroidX Bluetooth

Utilising the iOS Secure Enclave with your data


When cloud infrastructure is created using IaC with Terraform, how can you know it’s working as intended?

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-25-Streamlining-Infrastructure-as-Code-Testing-Harnessing-Terratest-Terraform-and-AWS-Go-SDKs-for-Automation/meme.webp>
<figcaption></figcaption>
</figure>

I can recall many times when I deployed services with Terraform only to find out it didn’t work as intended. I would then sometimes spend hours pecking through the cloud UI looking for a missing configuration or what could possibly cause the issue. Sometimes it would even be the same missing configuration for a previous Terraform deployment. You know, like those elusive Security Group's egress/ingress rules. Only if I had an automated way to test for configurations that should be set or services that should be active. Let me introduce unit and integration tests using [Terratest from Gruntwork](https://terratest.gruntwork.io/).

## Unit Tests

Unit tests are automated tests to ensure that a section of an application (known as the "unit") meets its design and functions as intended. With Terraform this can really only be done by deploying to a real environment. When deploying infrastructure, it is very rarely just a single unit, however more an integration test of a few to many services. For that reason, there is no true unit testing with IaC.

So how can this be done? How can we test our infrastructure in the smallest unit possible? Let’s take this simple infrastructure example.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-25-Streamlining-Infrastructure-as-Code-Testing-Harnessing-Terratest-Terraform-and-AWS-Go-SDKs-for-Automation/infra.webp>
<figcaption></figcaption>
</figure>

Testing this entire infrastructure would take too long and would have a high risk of failure. This is why relying on [end to end testing for infrastructure is not advised](https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html), and should be broken up into smaller unit and integration tests. In this example, we can do this by making sure each service is broken up into modules.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-25-Streamlining-Infrastructure-as-Code-Testing-Harnessing-Terratest-Terraform-and-AWS-Go-SDKs-for-Automation/infra-modules.webp>
<figcaption></figcaption>
</figure>

By testing each module we can deploy our infrastructure, validate it works, and then destroy it. Here is a snippet of code on performing this test on an AWS VPC with Terratest.

```
  defer terraform.Destroy(t, terraformOptions)
  terraform.InitAndApply(t, terraformOptions)
  outputs, err := TerraformOutputAll(t, terraformOptions)
  CheckIfError(err)
  validateVPC(t, outputs)
```

Let's walk through each line. Most of these lines have a `terraformOptions` these are options that are passed. These include the Terraform directory, and environment variables, or any Terraform variables to pass. They also may have a `t`, which is the [testing](https://pkg.go.dev/testing) package in Go.

1. In Golang, the defer keyword is used to delay the execution of a function or a statement until the nearby functions return. Meaning that the Terraform will destroy the deployed infrastructure once the deployment and validation is completed, successfully, or not.
2. The Terraform `init` and `apply` commands are run, which will deploy the infrastructure.
3. Outputs are injected into a variable to be used to validate our tests. Some examples are VPC ID, public subnets, and private subnets.
4. A wrapper to check for any errors.
5. Run Go tests to validate that the VPC is working as intended.

Here is an example of two tests. One that will verify that the public subnets are indeed public, and the other verifying that the privates are in fact private. Don't focus too much on the code, what I want to point out is that we can take the Terraform outputs, and assert that they are public, or private by using the `assert.True` or `assert.False`. Meaning if a private subnet that was created was public, it would fail the test.

```
  // Verify if the network that is supposed to be public is really public
  for _, v := range outputs.AWSVPCPublicSubnets {
    assert.True(
      t,
      aws.IsPublicSubnet(
        t,
        string(v),
        awsRegion,
      ),
    )
  }

  // Verify if the network that is supposed to be private is really private
  for _, v := range outputs.AWSVPCPrivateSubnets {
    assert.False(
      t,
      aws.IsPublicSubnet(
        t,
        string(v),
        awsRegion,
      ),
    )
  }
```

When I run the tests on the VPC and they all pass, I would get the results like so.

```
=== RUN   TestVPC/Subnets
--- PASS: TestVPC (7.11s)
    --- PASS: TestVPC/Subnets (0.93s)
PASS
ok      example 7.474s
```

The above example is testing the subnets within a VPC. Which, I would call an integration test. since our VPC module includes an option to create subnets, the unit in question is the module itself. Before we go into what integration tests would look like, let's go over the stages in Terratest.

## Terratest Stages

Terratest has a helper function `RunTestStage`. This help function is used as a variable as `stage`. The quoted text is the name of the stage, which we can now skip by passing an environment variable. If we wanted to deploy this VPC, however, didn't want to destroy it after the run is completed, we can export the env var `SKIP_destroy_vpc`. This is important to include in the integration tests to avoid destroying infrastructure that was created in a separate module and needed for other services.

```
	stage := test_structure.RunTestStage

	defer stage(t, "destroy_vpc", func() {
		DestroyTerraform(t, terraformOptions)
	})

	stage(t, "apply_vpc", func() {
		outputs = ApplyTerraform(t, terraformOptions)
	})

	stage(t, "validate_vpc", func() {
		validateVPC(t, outputs)
	})
```

The output now shows that the `destroy_vpc` was skipped:

```
The 'SKIP_apply_vpc' environment variable is not set, so executing stage 'apply_vpc'.
applying...
The 'SKIP_validate_vpc' environment variable is not set, so executing stage 'validate_vpc'.
=== RUN   TestVPC/Subnets
The 'SKIP_destroy_vpc' environment variable is set, so skipping stage 'destroy_vpc'.
--- PASS: TestVPC (7.59s)
    --- PASS: TestVPC/Subnets (0.96s)
PASS
ok      example 7.925s
```

## Integrations Tests

Here is a real-world example of how integration tests would work. We want to deploy an ECS cluster, and we want to make sure that the container on the service is running, and that the ingress/egress security group rules for the ALB and ECS are set correctly. We already deployed other modules needed and left them deployed by using the `SKIP_destroy_foo` environment variables. Now running the tests will skip any stage name that matches the environment variables, in this example `SKIP_destroy_ecs`.

```
	stage := test_structure.RunTestStage

	var outputs TerraformOutputs

	defer stage(t, "destroy_ecs", func() {
		DestroyTerraform(t, terraformOptions)
	})

	stage(t, "apply_ecs", func() {
		outputs = ApplyTerraform(t, terraformOptions)
	})

	stage(t, "validate_ecs", func() {
		validateECS(t, outputs)
	})

	stage(t, "integration_ecs", func() {
		integrationECS(t, outputs)
	})
```

Within the `integration_ecs` we will be testing:

* VPC and subnets
* Security group ingress/egress rules
* ECS service running

```
The 'SKIP_apply_ecs' environment variable is not set, so executing stage 'apply_ecs'.
apply...
The 'SKIP_validate_ecs' environment variable is set, so skipping stage 'validate_ecs'.
The 'SKIP_integration_ecs' environment variable is not set, so executing stage 'integration_ecs'.
=== RUN   TestECS/ECS_Cluster
=== RUN   TestECS/VPC
=== RUN   TestECS/Security_Groups
The 'SKIP_destroy_ecs' environment variable is set, so skipping stage 'destroy_ecs'.
--- PASS: TestECS (12.30s)
    --- PASS: TestECS/ECS_Cluster (0.28s)
    --- PASS: TestECS/VPC (0.99s)
    --- PASS: TestECS/Security_Groups (0.21s)
PASS
ok      example 12.718s
```

## Conclusion

I can now with confidence test deployed infrastructure using Terraform by using Terratest. I can have Terratest deploy in a separate sandbox environment, validate it works with set checks, and finally destroy the created infrastructure once done. A few takeaways I have from testing IaC with automation are to avoid end-to-end testing, make sure to set a bigger time out on the Go test command with `-timeout`, and spend time thinking of checks that you want to include in your tests.

**Note**

HashiCorp has switched from [MPL v2 license](https://www.mozilla.org/en-US/MPL/2.0/FAQ/) to a “Business Source License” (BSL). This will have an impact on Terratest going forward past Terraform version 1.5.5, so just make sure for the time being not to upgrade past that version if using Terratest. If interested, look into the [OpenTofu Linux Project](https://opentofu.org/), which has a forked Terraform, and is dedicated to open source.


_Article Photo by James von Hagel_


Streamlining Infrastructure as Code Testing: Harnessing Terratest, Terraform, and AWS Go SDKs for Automation

Responsive Web Design: Techniques and Best Practices


Recently, our team's attention shifted towards The Composable architecture. This is a very powerful framework to build iOS applications. The goal of this article is to show that you can build the whole app logic without creating any UI with the help of TCA and TDD. It will not cover TCA basics and will be focused more on showing testing capabilities of TCA.

### Theory walkthrough
#### TCA
If you are reading this article I would assume that you already have some knowledge about what is TCA. Anyway here below you can find a quick brief with a diagram about this architecture.
![TCA Diagram]({{ sitePrefix }}/assets/img/articles/2023-10-26-The-Composable-architecture-and-TDD/TCA-diagram.webp
The Composable Architecture is a framework that can be used to build state-driven, testable applications. In TCA, each feature is represented by a Store that holds the feature's State and Reducer function. Feature State is updated by sending an Action to the Reducer function. The Reducer function takes the current state and action as parameters and mutates the current state to produce a new state. The View observes the state and updates itself when the state changes. \
Additionally, the Store manages feature Environment dependencies (API client, database client, location manager, etc.) that can be invoked by the Reducer function, potentially producing Effects. Effects can, in turn, trigger another action to be sent to the Reducer. This simple flow ensures that feature data flows in a unidirectional manner and decouples the View from the business logic, which is crucial when developing a mobile application. \
Keeping this in mind, we can recognise one of the key advantages of TCA - Testability. TCA strictly separates the view from business logic, enabling it to be tested without the need to create any views or provide any runtime dependencies. \
If you want to dive into the TCA basics, here is a list of available resources that you can explore:
- [TCA Documentation](https://pointfreeco.github.io/swift-composable-architecture/main/documentation/composablearchitecture)
- [GitHub repo with CaseStudies](https://github.com/pointfreeco/swift-composable-architecture)
- [Pointfree videos](https://www.pointfree.co/collections/composable-architecture) 

#### TDD
I think that the best definition of what is TDD is given in <em>Test-Driven iOS Development with Swift</em> book by Dr. Dominik Hauser: 
> As the name suggests, in TDD, tests drive the development. This means that the developer writes code only because there is a test that fails. The tests dictate whether the code has to be written, and they also provide a measure when a feature is "done" - it is done when all tests for the feature pass.

The TDD flow is simple and consists of only three steps:
- **Red** - you start by writing a failing test. It needs to test a required feature of the software product that is not already implemented or an edge case that you want to make sure is covered.
- **Green** - in the green step, you write the simplest code that makes the test pass.
- **Refactor** - in the refactor step you improve the code. You remove duplication, extract common values, and so on. You should never skip this step and always think how you can improve your code. When you came up with an improvement, first make sure it's reflected in your test and then refactor an implementation. **Not vice-versa!**

### Practice
#### Testing State changes
Let's say I want to have a TextField where user will input a name of their friend. There also will be a Discover button that should be enabled only when input name is not empty.
##### ❌ Red:
First, let's start with implementing a failing test. Here, in the screenshot below, you can see how the TCA library conveniently displays failing tests. It highlights all state changes and fields that differ from the expected output.
![Failing test]({{ sitePrefix }}/assets/img/articles/2023-10-26-The-Composable-architecture-and-TDD/failing-test-screenshot.webp
##### ✅ Green:
Now let's make this test pass by implementing the logic:
```swift
public struct HomeFeature: Reducer {
    public struct State: Equatable {
        @BindingState var name: String = ""
        var discoverButtonDisabled: Bool = true
    }
    
    public enum Action: Equatable, BindableAction {
        case binding(BindingAction<State>)
    }
    
    public var body: some ReducerOf<Self> {
        BindingReducer()
        Reduce { state, action in
            switch action {
            case .binding(\.$name):
                state.discoverButtonDisabled = state.name.isEmpty
                return .none
            case .binding(_):
                return .none
            }
        }
    }
}
```
All good, now the test passes:
![Passing test]({{ sitePrefix }}/assets/img/articles/2023-10-26-The-Composable-architecture-and-TDD/passing-test-screenshot.webp
##### 📈 Refactor:
I want to refactor the `discoverButtonDisabled` boolean in this section. I anticipate that in the future, this Discover button will trigger an API call and be replaced by a `ProgressView`. To prepare for this, let's convert the boolean into an enum value. Again, first start by changing your test implementation and then move to the Feature. \
**Test:**
```swift
    func testNameInput() async {
        let store = TestStore(initialState: HomeFeature.State()) { // Create test store for HomeFeature
            HomeFeature()
        }
        
        // Send action to imulate user input
        await store.send(.binding(.set(\.$name, "Steave"))) {
            // Assert name was changed to 'Steave'
            $0.name = "Steave"
            // Assert that Discover button was enabled
            $0.discoverButtonState = .enabled // Compiler error will be raised.
        }
    }
```
**Implementation:**
```swift
public struct HomeFeature: Reducer {
    enum ButtonState {
        case enabled
        case disabled
        case loading
    }
    
    public struct State: Equatable {
        @BindingState var name: String = ""
        var discoverButtonState: ButtonState = .disabled
    }

    public enum Action: Equatable, BindableAction {
        case binding(BindingAction<State>)
    }

    public var body: some ReducerOf<Self> {
        BindingReducer()
        Reduce { state, action in
            switch action {
            case .binding(\.$name):
                state.discoverButtonState = state.name.isEmpty ? .disabled : .enabled
                return .none
            case .binding(_):
                return .none
            }
        }
    }
}
```


#### Testing effects
In TCA, Effects play a big role, and it is crucial to be able to test them. Let's take a look how this is done. The logic is that when user presses the Discover button, an API request is fired in order to get the data to be shown. If the API request is successful, then a sheet showing the data should be opened. Otherwise, it should show an alert with an error message.

**Successfull response**
```swift
    func testDiscoverWishesTapped() async {
        let store = TestStore(initialState: HomeFeature.State()) {
            HomeFeature()
        } withDependencies: {
            $0.apiClient = .testValue // Use mock API client
        }
        store.exhaustivity = .off(showSkippedAssertions: true) // Test only assertions
        await store.send(.discoverWishesTapped) // Nothing happens
        
        // User inputs name
        await store.send(.binding(.set(\.$name, "Steave"))) 
        
        // expected wishes
        let expected = try! await APIClient.mock.requestWishes()
        
        // User taps discover
        await store.send(.discoverWishesTapped) {
            $0.discoverButtonState = .loading
        }
        // Receive wishes and open sheet
        await store.receive(.wishesResponse(.success(expected)), timeout: .seconds(0.1)) {
            $0.discoverButtonState = .enabled
            $0.destination = .submitWish(SubmitWishFeature.State(name: "Steave", wishes: expected.wishes))
        }
        
        // Check if we actually created scoping between two features
        // by sending an action to child store through the parant
        await store.send(.destination(.presented(.submitWish(.selectWish(0))))) {
            $0.$destination[case: /HomeFeature.Destination.State.submitWish]?
                .selectedWish = .init(id: "mock1", title: "Health", iconName: "heart.square")
        }
    }
```
**Failing response**
```swift
    func testFailingDiscoverWishes() async {
        let store = TestStore(initialState: HomeFeature.State(name: "Steave")) {
            HomeFeature()
        } withDependencies: {
            $0.apiClient = .failing // set API client to failing
        }
        
        // Request is triggered
        await store.send(.discoverWishesTapped) {
            $0.discoverButtonState = .loading
        }
        
        // Failed response, expect that apiError will be shown
        await store.receive(.wishesResponse(.failure(APIError.callUnimplemented)), timeout: .seconds(0.1)) { 
            $0.discoverButtonState = .enabled
            $0.destination = .alert(.apiError)
        }
        
        // Test that user can close the alert
        await store.send(.destination(.presented(.alert(.ok)))) {
            $0.destination = nil
        }
    }
```
Now let's focus on key TCA features that are used here:
- `.receive()` - API response produces an effect with the result, wich has to be received in the test. This method will do it for you and you will be able to make assertions agains the new State. On the other hand if there is some Effect that was not received, the test will fail.
- `withDependency` - Setting this parameter is a great way to control dependencies you provide to a test. In this example I use different API client implementations. One for mock another one for failing.
- `exhaustivity` - When this parameter is set to .on, the test framework checks whether all possible code paths and conditions in your application have been tested. If you skip an assertion for some state value, the test will fail. In cases where it's set to .off, only the assertions you make will be tested.
- `destination` - This is a `@PresentationState` value which determines a state of presented sheet/alert. Nothing is presented when it's set to `nil`. In addition, it tests whether parent and child domains can exchange actions between each other by sending the `.selectWish` action from the parent to the child. In TCA, this is basically called scoping.

#### Testing Navigation stack
As you might know, the TCA framework provides an out-of-the-box solution for stack navigation with the help of `StackState`. Essentially, it holds the states of all child domains and allows you to push and pop them, as well as send actions between them. Because it's state-driven, you can easily test it. \
In the following example, the user is presented with a list of wishes. The user is able to select only one wish and after pressing the Proceed button, they are taken to results screen. Let's take a look how this behaviour can be tested.
```swift
    func testProceedButtonTapped() async {
        let store = TestStore(initialState: SubmitWishFeature.State(name: "Steave", wishes: wishes)) {
            SubmitWishFeature()
        }
        // Nothing happens if nothing is selected
        await store.send(.proceedButtonTapped)
        
        // User selects a wish
        await store.send(.selectWish(0)) {
            $0.buttonState = .enabled
            $0.selectedWish = .init(id: "mock1", title: "Health", iconName: "heart.square")
        }
        
        // Request is sent to save selection on backend side
        await store.send(.proceedButtonTapped) {
            $0.buttonState = .loading
        }
        
        // Receive successfull response and open Result screen.
        await store.receive(.saveWishResponse(.success(.init())), timeout: .seconds(0.1)) {
            $0.buttonState = .enabled
            $0.path[id: 0] = .result(.init(name: "Steave", wish: .init(id: "mock1", title: "Health", iconName: "heart.square")))
        }
    }
```
Once response is received a new screen is pushed to navigation stack. We can test this by getting the first element from the `path` and asserting against its state. 

Moreover, with TCA you can test the navigation by sending push/pop as well as child domain actions directly. Check out the following example:
```swift
    func testResultNavigation() async {
        let store = TestStore(initialState: SubmitWishFeature.State(name: "Steave", wishes: wishes)) {
            SubmitWishFeature()
        }
      
        // Directly send push action to the store
        await store.send(.path(.push(id: 0, state: .result(ResultWishFeature.State(name: "Steave", wish: .init(id: "mock1", title: "Health", iconName: "heart.square")))))) {
            $0.path[id: 0] = .result(ResultWishFeature.State(name: "Steave", wish: .init(id: "mock1", title: "Health", iconName: "heart.square")))
        }
        
        // Send action into child domain and assert State changes.
        await store.send(.path(.element(id: 0, action: .result(.shareInMyProfileTapped)))) {
            $0.path[id: 0, case: /SubmitWishFeature.Path.State.result]?.isSharedInMyProfile = true
        }
        
        // Pop from child domain and assert navigation stack is empty
        await store.send(.path(.popFrom(id: 0))) {
            $0.path = StackState()
        }
    }
```

#### Conclusion
Here we've covered testing of the most esential parts of any iOS application:
- State changes by user input or user interaction
- State changes by Effects
- Dependency management inside tests
- Presentation of sheets and alerts
- Navigation stacks testing

All of these aspects are covered by the TCA framework out-of-the-box. Combined with a strong separation of business logic from the UI, it ensures the full testability of iOS apps. Moreover, TCA employs scoping, enabling the exchange of data and actions between different parts of the application. Essentially, it transforms the entire app into a global state, consisting of states from standalone features. This allows developers to test comprehensive scenarios involving different features rather than small, isolated parts of the app. This capability of TCA is crucial when employing Test-Driven Development for application development.

The Composable architecture and TDD. Can you make it work?

Last time we looked at [how to get started with Swift Charts]({{ baseurl }}/post/2023/10/23/Getting-started-with-Swift-Charts). Let's pick up where we left off.

We have the data, we have three ChartTypes, which show temperature (line chart), precipitation (bar chart) and wind (double line chart) and we have customized the axes.

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/5.webp)

## Data pre-processing

Looking at the wind chart, with the double line, it is a bit difficult to make sense of it, there's just too much data. So let's see what we can do about it. We have one measurement every ten minutes for 24 hours, so in total, we have 144 measurements. Plotting 144 data points for the average wind speed and for the maximum wind speed on the same chart makes it a bit crowded. But maybe we don't need all those data points. Let's have a look at how we can turn a data set with 144 points into one with only 24, as we want one measurement every hour for the wind.

One way to do it could be to only pick every 6th measurement, and just ignore the rest. But this new dataset could end up showing a different trend than the initial one and is not a good downsampling representation of the original data set. We need to do this in such a way that for every 6 data points in the original data set, we have only one data point, and this new one is an accurate representation or summarization of the original 6. This process is called downsampling.

So let's create a function that takes the original data set and returns the downsampled one:

```swift
func downsample(originalArray: [WeatherDataPoint]) -> [WeatherDataPoint] {
  var downsampledArray: [WeatherDataPoint] = []
  // .. We'll add some stuff here
  return downsampledArray
}
```

We'll use Swift's `stride` function because it's perfect for this use case. The `stride` function takes a `start`, an `end` and a `stride`, and returns a sequence from `start` to, but not including, `end`, stepping by `stride`. So for example, `stride(from: 0, to: 10, by: 2)` will be `[0, 2, 4, 6, 8]`, and `stride(from: 0, to: 10, by: 3)` will be `[0, 3, 6, 9]`. We want to create slices of the original data source, of size 6.

```swift
for i in stride(from: 0, to: downsampledArray.count, by: 6) {
      let endIndex = min(i + 6, downsampledArray.count)
      let slice = Array(self[i..<endIndex])
      let aggregatedValue: WeatherDataPoint
      // .. Figure out how to calculate the aggregated value
      downsampledArray.append(aggregatedValue)
  }
```

But what is the aggregated value? That's interesting, right? Because it depends. Our data set measures wind average speed, and wind maximum speed. What's a good aggregated value for 6 average speed data points? Well, it's the average of those. What about the maximum speed? It's the maximum of those. Because the average wind speed in an hour is the average of the wind speed in the first 10 minutes, the one in the following 10 minutes, and so on (6 times). And similarly for the maximum speed.

So let's pass the aggregation function as a parameter, and redefine our method like this:

```swift
func downsample(originalArray: [WeatherDataPoint], aggregationFunction: ([WeatherDataPoint]) -> WeatherDataPoint) -> [WeatherDataPoint] {
    var downsampledArray: [WeatherDataPoint] = []

    for i in stride(from: 0, to: originalArray.count, by: 6) {
        let endIndex = min(i + 6, originalArray.count)
        let slice = Array(originalArray[i..<endIndex])
        let aggregatedValue = aggregationFunction(slice)
        downsampledArray.append(aggregatedValue)
    }
    return downsampledArray
}
```

To make it even nicer, we can make this method an extension on Array and pass the `step` (the length of the slice) as a parameter too:

```swift
extension Array where Element == WeatherDataPoint {
    func downsample(step: Int, aggregationFunction: ([WeatherDataPoint]) -> WeatherDataPoint) -> [WeatherDataPoint] {
        var downsampledArray: [WeatherDataPoint] = []

        for i in stride(from: 0, to: self.count, by: step) {
            let endIndex = Swift.min(i + step, self.count)
            let slice = Array(self[i..<endIndex])
            let aggregatedValue = aggregationFunction(slice)
            downsampledArray.append(aggregatedValue)
        }

        return downsampledArray
    }
}
```

We can now use this method to downsample the wind data series, so we add to the `StationMeasurements` the following:

```swift
var downsampledWindSeries: [WeatherDataPoint] {
  return windSeries.downsample(step: 6) { slice in
    let count = slice.count
    let values = slice.map { $0.measuredValue }
    let secondaryValues = slice.compactMap { $0.secondaryMeasuredValue }
    return WeatherDataPoint(dateTime: slice.safelyAccessElement(at: count/2)?.dateTime ?? slice[0].dateTime,
                            measuredValue: values.reduce(0, +) / Double(count),
                            secondaryMeasuredValue: secondaryValues.max())
        }
    }
```

Where `safelyAccessElement` is a helper method which gives us the element at one index, or nil if the index is out of bounds.

```swift
private extension Array {

    func safelyAccessElement(at index: Int) -> Element? {
        guard indices.contains(index) else {
            return nil
        }

        return self[index]
    }

}
```

Now we have managed to turn our data set from 144 points to 24 points. We can use this one to draw the wind chart instead, and the chart immediately looks better.

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/6.webp)

## Interactive charts

Now that the charts look better, let's make them even more useful: let's show the value for a precise measurement when the user taps on a point in the chart.

We want to detect when the user taps on the chart, show an indicator of the point they tapped, and show the value at that point in a label above the chart. So we add a `.chartOverlay` which detects the taps and drags to our chart like this:

```swift
.chartOverlay { proxy in
    GeometryReader { geo in
        ZStack(alignment: .topLeading) {
            Rectangle().fill(.clear).contentShape(Rectangle())
                .gesture(
                    DragGesture(minimumDistance: 0.0)
                        .onChanged { value in
                            switch chartType {
                            case .temperature(let data), .precipitation(let data), .wind(let data):
                                selectedElement = findElement(in: data, location: value.location, proxy: proxy, geometry: geo)
                            }
                        }
                        .onEnded { _ in
                            selectedElement = nil
                        }

                )
            if let selectedElement {
                LollipopView(chartType: chartType, selectedElement: selectedElement, chartProxy: proxy, geometryProxy: geo)
            }
        }
    }
}
```

We want to keep the currently selected element in the `selectedElement` `@State` property, which can be nil if no element is currently selected (if the user isn't currently tapping or dragging their finger on the chart). If the user taps on the chart and an element is selected, we show an indicator of that (vertical line from the X axis to the chart point where the user tapped, and a rounded circle at its top; we call this `LollipopView`)

For finding the element on which the user tapped, we have to translate the touch point from screen coordinates to chart coordinates and find the closest data point to that. Here is the `findElement` method:

```swift
private func findElement(in data: [WeatherDataPoint], location: CGPoint, proxy: ChartProxy, geometry: GeometryProxy) -> WeatherDataPoint? {
    let relativeXPosition = location.x - geometry[proxy.plotAreaFrame].origin.x
    if let date = proxy.value(atX: relativeXPosition) as Date? {

        // Find the closest element.
        var minDistance: TimeInterval = .infinity
        var index: Int?
        for dataIndex in data.indices {
            let nthDataDistance = data[dataIndex].dateTime.distance(to: date)
            if abs(nthDataDistance) < minDistance {
                minDistance = abs(nthDataDistance)
                index = dataIndex
            }
        }
        if let index {
            return data[index]
        }
    }
    return nil
}
```

When we have a selected element, we want to show a visual indicator of the tapped point, so here's our LollipopView:

```swift
struct LollipopView: View {
    let chartType: ChartType
    let selectedElement: WeatherDataPoint
    let chartProxy: ChartProxy
    let geometryProxy: GeometryProxy

    var body: some View {

        let dateInterval = Calendar.current.dateInterval(of: .minute, for: selectedElement.dateTime)!
        let startPositionX1 = chartProxy.position(forX: dateInterval.start) ?? 0

        let lineX = startPositionX1 + geometryProxy[chartProxy.plotAreaFrame].origin.x
        let chartAreaHeight = geometryProxy[chartProxy.plotAreaFrame].maxY
        let plotLineY = chartAreaHeight - (chartProxy.position(forY: selectedElement.secondaryMeasuredValue ?? selectedElement.measuredValue) ?? 0)

        // The vertical line
        Rectangle()
            .fill(.gray)
            .frame(width: 2, height: plotLineY)
            .position(x: lineX, y: chartAreaHeight - plotLineY / 2)

        // The symbol shown at the point we've tapped on the graph
        Circle()
            .foregroundColor(.blue)
            .frame(width: 16, height: 16)
            .overlay(
                Circle()
                    .stroke(Color.white, lineWidth: 4)
            )
            .position(x: lineX, y: chartProxy.position(forY: selectedElement.measuredValue) ?? 0)

        // If we're on the wind data chart, show the diamond on the gust line too
        if case .wind = chartType {
            Rectangle()
                .rotation(Angle(degrees: 45))
                .aspectRatio(1.0, contentMode: .fit)
                .foregroundColor(.red)
                .frame(width: 13, height: 13)
                .overlay(
                    Rectangle()
                        .rotation(Angle(degrees: 45))
                        .stroke(.white, lineWidth: 4)
                        .aspectRatio(1.0, contentMode: .fit)
                )
                .position(x: lineX, y: chartProxy.position(forY: selectedElement.secondaryMeasuredValue ?? 0) ?? 0)
        }
    }
}
```

On top of what we mentioned, for the wind chart which has 2 line charts, we want to show different symbols on each line: a circle on the average speed and a diamond on the gust. That's why we added the last block of code in the LollipopView.

Printing the latest measurement or the currently selected measurements above the chart should be pretty straightforward, as long as you make sure you format the data correctly.

And here's the LollipopView in action. We just made our charts interactive!

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/7.webp)
<center>{{ html5-link="/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/8.mov" }}</center>

Resources:
1. [Swift-Charts-Examples](https://github.com/jordibruin/Swift-Charts-Examples)

_Article photo generated with [Midjourney](https://www.midjourney.com/)_


Getting started with Swift Charts - part 2

Let's have a look at [Swift Charts](https://developer.apple.com/documentation/Charts) and see how we can use it to plot weather data.

Apple introduced Swift Charts in iOS 16, and improved it in iOS 17, adding, among others, new chart types. If your app still needs to support iOS 15 or lower versions, you can use a 3rd party library, like [Daniel Gindi's Charts](https://github.com/danielgindi/Charts). In this demo project, we'll focus on the initial release of Swift Charts and well target iOS 16.


## Data
First, let's find some data to plot. We'll be looking at weather data, and, for the sake of simplicity, we'll just load locally some JSON from [this file](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/StationData.json). The JSON looks like this:

```json
{
    "name": "Københavns Lufthavn",
    "latitude": 55.6139,
    "longitude": 12.6455,
    "temperatureSeries": [{
        "dateTime": "2023-09-21T13:10:00Z",
        "measuredValue": 21.9
    }, ...
    ],
    "precipitationSeries": [{
        "dateTime": "2023-09-21T13:00:00Z",
        "measuredValue": 1.4
    }, ...
    ],
    "windSeries": [{
        "dateTime": "2023-09-21T13:10:00Z",
        "measuredValue": 6.2,
        "secondaryMeasuredValue": 8.2
    },
    ]
}
```

We have the details of a station, with its name, and position, and then we have 3 series of data, one for temperature (measured in Celsius degrees), one for precipitation (measured in millimetres) and one for wind, which contains the value of the average wind and the value of the gust (max wind speed) for that particular time. All entries in the data series have a `dateTime`. For temperature and wind, we have 144 data points, corresponding to one measurement every 10 minutes, and for precipitation we have 24 data points, corresponding to a measurement every hour.

Let's start a new Xcode project, using SwiftUI, and let's add the JSON file to the project and the target.

We want to parse this data, so we're creating our model. The purpose of this article is to play with Swift Charts; there are other, possibly better-suited ways of modelling the data, but for now, let's create a WeatherDataPoint struct.

```swift
struct WeatherDataPoint: Codable, Identifiable {
    let dateTime: Date
    let measuredValue: Double
    let secondaryMeasuredValue: Double?

    var id: Date {
        dateTime
    }
}
```

And a StationMeasurements struct:

```swift
struct StationMeasurements: Codable {
    let name: String
    let latitude: Double
    let longitude: Double
    let temperatureSeries: [WeatherDataPoint]
    let precipitationSeries: [WeatherDataPoint]
    let windSeries: [WeatherDataPoint]
}
```

And let's parse the data from the JSON file into a mock:

```swift
extension StationMeasurements {
    static var mock: StationMeasurements {
        let url = Bundle.main.url(forResource: "StationData", withExtension: "json")!
        let data = try! Data(contentsOf: url)
        let decoder = JSONDecoder()
        decoder.dateDecodingStrategy = .iso8601
        let jsonData = try! decoder.decode(StationMeasurements.self, from: data)
        return jsonData
    }
}
```

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/1.webp)

## Basic Charts
We now have the data in our project, we can go ahead and start playing with it.
We want to have three types of charts, a line chart to show the temperature, a bar chart to show precipitations, and another one with two lines on the same chart, so we can see the average and maximum wind on the same graph.

Let's describe that in code, by making an enum with a case for each chart type, and the data as an associated value.

```swift
enum ChartType {
    case temperature(temperatureData: [WeatherDataPoint])
    case precipitation(precipitationData: [WeatherDataPoint])
    case wind(windData: [WeatherDataPoint])
}
```

Let's start by creating the temperature chart. We'll make a new SwiftUI View, called ChartView. It will have a `ChartType`, which we'll also use to pass the data in through the associated value of the enum cases. The temperature chart will be a line chart, so inside the `Chart` we'll create a `LineMark`. We can then configure the line style, color and interpolation method, to make it prettier.

```swift
struct ChartView: View {
    let chartType: ChartType

    var body: some View {
        switch chartType {
        case .temperature(let temperatureData):
            Chart {
                ForEach(temperatureData) { element in
                    LineMark(
                        x: .value("dateTime", element.dateTime),
                        y: .value("temperature", element.measuredValue)
                    )
                    .lineStyle(StrokeStyle(lineWidth: 4))
                    .foregroundStyle(Color.blue)
                    .interpolationMethod(.cardinal)
                }
            }
        case .precipitation(let precipitationData):
            EmptyView()
        case .wind(let windData):
            EmptyView()
        }

    }
}
```

For the other two chart types, we can return `EmptyView()` for now. Here is the line chart showing the temperature data:

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/2.webp)

Let's continue and, similarly, create the bar chart. This time, we have to use the `BarMark`, but mostly everything else remains the same; we can remove the `interpolationMethod`, which doesn't make sense for a bar chart, and just change the identifiers.

```swift
case .precipitation(let precipitationData):
    Chart {
        ForEach(precipitationData) { element in
            BarMark(
                x: .value("dateTime", element.dateTime),
                y: .value("precipitation", element.measuredValue)
             )
              .lineStyle(StrokeStyle(lineWidth: 4))
              .foregroundStyle(Color.blue)
         }
    }
```

The wind chart will be a bit more interesting, it will consist of two line charts. So on the same `Chart`, we will have two `LineMark`s. The main difference from the temperature chart is that, for each LineMark, we will have to specify the `series` argument as well, so they will be plotted as individual series. We want the max wind speed to be plotted as a dashed line, so we also specify the `dash` parameter in the lineStyle.

```swift
case .wind(let windData):
    Chart {
        ForEach(windData) { element in
            LineMark(
                x: .value("dateTime", element.dateTime),
                y: .value("average wind", element.measuredValue),
                series: .value("average", "average wind")
            )
            .lineStyle(StrokeStyle(lineWidth: 4))
            .foregroundStyle(Color.blue)
            .interpolationMethod(.cardinal)

            if let secondaryMeasuredValue = element.secondaryMeasuredValue {
                LineMark(
                    x: .value("dateTime", element.dateTime),
                    y: .value("max wind", secondaryMeasuredValue),
                    series: .value("gust", "gust")
                )
                .lineStyle(StrokeStyle(lineWidth: 4, dash: [6]))
                .foregroundStyle(Color.red)
                .interpolationMethod(.cardinal)
            }
        }
    }
```

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/3.webp)

Now, we have the data plotted, but we would like to configure the axes. We want the vertical axis to be to the left of the chart, and we want to be able to change the font of the label on the vertical axes. Adding this code after the `Chart {...}` does the trick.

```swift
.chartYAxis {
    AxisMarks(position: .leading) {
        AxisValueLabel()
        AxisGridLine(stroke: StrokeStyle(lineWidth: 0.5))
    }
}
```

But that only customizes one of the three charts, and we don't like to repeat code.

![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/4.webp)

So what we want now is to extract the Chart content, and use only one `Chart`, to be able to customize the axes the same way for all types of charts without repeating code.

Let's create the chart contents for our three charts.

```swift
struct LineChartViewContent: ChartContent {
    let data: [WeatherDataPoint]

    var body: some ChartContent {
        ForEach(data) { element in
            LineMark(
                x: .value("dateTime", element.dateTime),
                y: .value("temperature", element.measuredValue)
            )
            .lineStyle(StrokeStyle(lineWidth: 4))
            .foregroundStyle(Color.blue)
            .interpolationMethod(.cardinal)
        }
    }
}

struct BarChartViewContent: ChartContent {
    let data: [WeatherDataPoint]

    var body: some ChartContent {
        ForEach(data) { element in
            BarMark(
                x: .value("dateTime", element.dateTime),
                y: .value("precipitation", element.measuredValue)
            )
            .lineStyle(StrokeStyle(lineWidth: 4))
            .foregroundStyle(Color.blue)
        }
    }
}

struct DoubleLineChartViewContent: ChartContent {
    let data: [WeatherDataPoint]

    var body: some ChartContent {
        ForEach(data) { element in
            LineMark(
                x: .value("dateTime", element.dateTime),
                y: .value("average wind", element.measuredValue),
                series: .value("average", "average wind")
            )
            .lineStyle(StrokeStyle(lineWidth: 4))
            .foregroundStyle(Color.blue)
            .interpolationMethod(.cardinal)

            if let secondaryMeasuredValue = element.secondaryMeasuredValue {
                LineMark(
                    x: .value("dateTime", element.dateTime),
                    y: .value("max wind", secondaryMeasuredValue),
                    series: .value("gust", "gust")
                )
                .lineStyle(StrokeStyle(lineWidth: 4, dash: [6]))
                .foregroundStyle(Color.red)
                .interpolationMethod(.cardinal)
            }
        }
    }
}
```

## Customizing the axes

And now we can modify the ChartView to use the newly created ChartContents, and then we can customize the axes once for all charts.

```swift
struct ChartView: View {
    let chartType: ChartType

    var body: some View {
        Chart {
            switch chartType {
            case .temperature(let temperatureData):
                LineChartViewContent(data: temperatureData)
            case .precipitation(let precipitationData):
                BarChartViewContent(data: precipitationData)
            case .wind(let windData):
                DoubleLineChartViewContent(data: windData)
            }
        }
        .chartYAxis {
            AxisMarks(position: .leading) {
                AxisValueLabel()
                AxisGridLine(stroke: StrokeStyle(lineWidth: 0.5))
            }
        }
    }
}
```

We also want to customize the horizontal axes. On this one, we display dates. Since the measurements are for 24 hours, and the space on the axis is quite limited, we're only interested in the time. We print one label every 4 hours and only print the time. We also show a grid line every 4 hours.

```swift
.chartXAxis {
    AxisMarks(values: .stride(by: .hour, count: 4)) { value in
        if let date = value.as(Date.self) {
            let hour = Calendar.current.component(.hour, from: date)
            AxisValueLabel {
                Text(date, style: .time)
            }
            AxisGridLine()
        }
    }
}
```

So here's how far we've come:
![](/assets/img/articles/2023-10-23-Getting-started-with-Swift-Charts/5.webp)

Where can we go from here? It would be nice if we could tap on the chart to see the exact measurement value, right? And maybe we can do something about that wind chart, which looks a bit crowded. We're looking at that in [part 2]({{ baseurl }}/post/2023/10/23/Getting-started-with-Swift-Charts-part-2).

_Article photo generated with [Midjourney](https://www.midjourney.com/)_


Getting started with Swift Charts - part 1

Strengthening our AI knowledge on mobile apps

[Spatial](https://spatial.io/) is a metaverse platform that allows anyone to create immersive 3D spaces, which can be instantly shared to the Web, iOS, Android, and VR, and explored by others. The Spatial Creator Toolkit, powered by Unity, allows you to add even more interactivity to the experience, without needing to code. This tutorial will guide you through the process of creating and uploading your first Spatial experience using the Creator Toolkit on Unity.

### Before we get started:
* Create an account with [Spatial](https://spatial.io/signup)
* Install [Unity version 2021.3.21](https://unity.com/releases/editor/whats-new/2021.3.21) and install the “WebGL Build Support” module with it. If you don’t yet have Unity Hub installed, you will need to [install it first](https://unity.com/download)
* Download the [Spatial Unity Starter Template](https://drive.google.com/file/d/1AAdOCsfh-E1OlIqAwHOgBjVBxG4CYCyN/view?usp=sharing)
* Download this [3D golf ball](https://sketchfab.com/3d-models/golf-ball-5f158949c1084575abf02437f6b43028) in .obj file format 
* Download this [Monstarlab Badge](https://drive.google.com/file/d/1qQeiCBRPuWM8B2sfJdJYPeNmhBUHVU-K/view?usp=sharing)

Start by opening the Spatial Starter template project in Unity. You will be prompted to update to the latest Spatial SDK. Click 'yes' and allow the project to load. We will need to configure our Spatial credentials first. Click on the 'Spatial SDK' tab in the header > Account. This will open the Spatial Portal window (which we will keep coming back to). 

Click on "Get Login Token", which will redirect you to your browser where you can copy your token directly to your clipboard. Once you copy it, come back to Unity and click on "Paste Login Token". Now you should be ready and set up to start working!

![Account Config]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/AccountConfig.webp

In the Project window, search for "GolfCourseDriving" and open this scene.

![Golf Course Driving]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Golf_Blue.webp)

In the Spatial Portal window, under the 'Config' tab, set 'Active Package' to 'Space - Golf Course'. Now that we've set this golf course environment as our active scene, we can go ahead and test this scene in Spatial by clicking on 'Test Active Scene' at the top of the window.

![Active Package]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Config.webp

If this is your first time testing a scene on Spatial, it will take some time to load, but subsequent tests will be much quicker. Once it's done, your sandbox environment will open on your browser.

![Sandbox Environment]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Spatial_Course.webp)

You can walk around using your arrow keys. Walk up to the golf cart and click [F] to drive it. Now your arrow keys will drive the golf cart around the scene! This is already pretty fun, but let's go back to Unity and see how we can add more interactivity to this scene.

Back in Unity, let's change out the default blue sky. Open the lighting window from 'Window' > 'Rendering' > 'Lighting'. Under the 'Environment' tab, click on the small circle next to 'Skybox Material'. This will let you search through your project to replace the material. Search for 'Epic_GloriousPink' and select it.

![Lighting]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Lighting.webp

You will notice instantly that this also changes the ambient light of the scene to a more pink-ish hue. That's because the source of the environment lighting is currently being pulled from the sky, which is what happens in real life. However, if you'd like to control the mood of the scene yourself, you can choose a color from 'Lighting' > 'Environment lighting' > 'Source' > 'Color'. Personally, I like the pink sunset!

![Pink Sunset]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Golf_Pink.webp)

Now, let's import the 3D model of the golf ball into the project. You can do this by opening the root of your Unity project in your File explorer/Finder and pasting the 3D model folder inside the 'Assets' folder. For more information on importing 3D objects and how to deal with materials and textures, you can read more [here.](https://docs.unity3d.com/Manual/ImportingModelFiles.html) 


From the Project window, search for "Trigger (Collectable)" and drag this prefab directly into your scene. Set its position in the Transform component to (X: 522, Y: 4.48, Z: 654). If this is your first time using Unity, you can read [here](https://docs.unity3d.com/Manual/SceneViewNavigation.html) to learn how to navigate the scene view. 

If you look at the 'Spatial Trigger Event' component, you can see a few things that will happen when the player walks into the radius of this object.

![Trigger Collectable]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Trigger1.webp

It's listening for 'Local Avatar', which is any player in Spatial. Once a player enters this trigger, 'On Enter Event' is called and the following happens:
- A coin sound effect is played
- The coin game object is disabled
- A particle effect plays
- The 'Spatial Trigger Component' is disabled

All of these happening in unison gives the impression of the player "collecting" the coin when they walk over it.

Let's switch out the coin model to the golf ball we imported earlier. Do this by disabling the child object "Coin" and dragging in the golf ball model as a child to the 'Trigger (Collectable)' game object. Scale and position it so it's roughly at the same size and position as the coin. 

![Golf ball on course]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Ball_Course.webp)

Make sure to switch out the coin for the golf ball in the 'On Trigger Event', so the new 3D model is disabled when collected.

![On Trigger Event]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Trigger2.webp

Now, let's set up a Quest. A Quest is a set of tasks the player needs to complete in order to fulfil a certain goal or objective. We can also reward players when they finish a quest, but more on that later.

Create an empty game object in the scene hierarchy and name it 'Golf Quest'. Click 'Add Component' > 'Spatial Quest'. Give your Quest a name and description. Enable the options as shown below:

![Spatial Quest]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Quest1.webp

Let's add a task under the Tasks list.

![Task List]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Task1.webp

This task shows us that we would need to collect 5 golf balls to complete this task, but we need  some way of tracking that a golf ball has been collected. Back in the 'Golf Ball' game object, in the 'Spatial Trigger Event Component' > 'On Enter Event', add a 'Quest Event'. Select the Quest you created, and choose 'Add Task Progress' and 'Golf balls collected'.

![Quest Event]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/QuestEvent.webp

Now, when a player walks over this golf ball, it will update the task progress on the Quest. On the Quest game object, we said they would need to collect 5 golf balls, so go ahead and duplicate the golf ball 4 more times and place them around the environment as you like. Once you're done, go ahead and test the active scene to try it out yourself on Spatial!


If you run around and collect the balls, the quest gets updated on the bottom left of the screen. However, if you try driving a golf cart to collect it, you'll notice it's unable to pick it up. Let's fix this.

Back in Unity, select a GolfCart game object. You'll see it's on the 'Vehicle' layer. If you check any of the golf ball triggers, it's listening for a 'LocalAvatar', which would be the player. On the golf cart, switch out its layer to 'AvatarLocal'. The editor will ask if you want to change the layer of all its children objects too. Select 'No, this object only'.

![Game object layers]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Layer.webp

Now, we need to add a box collider around the cart to define the boundary of what needs to collide with the golf ball to trigger its event. On the cart object, click 'Add Component' > 'Box Collider'. Set 'IsTrigger' to true. Edit the box collider to be bigger than the cart.

![Box Collider]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Cart_Collider.webp)

Go ahead and repeat these steps for all the carts. Test the scene again in your sandbox environment and you'll find you can drive the carts around to collect the golf balls now!

![Collect golf balls]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Cart_Collect.webp)


Finally, to reward the players for finishing the quest, we need to publish our space first (it can be set to private). In the Spatial Portal > Config tab, select 'Create New World'and then select 'Publish'. You'll receive an email once your space is successfully published. After that, we can set up a badge in [Spatial Studio](https://www.spatial.io/studio/worlds).

Select your world, and then select Badges > + New Badge. Here, you can upload the Monstarlab badge that you downloaded at the start of this blog (or feel free to create your own!). Give your badge a name and description.

![Badge Description]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/Badge.webp

Now you can return to Unity, and go back to your Quest object. Add a 'Quest Reward' and select 'Badge'. Copy and paste the 'Id' from your badge in Spatial Studio.

![Badge ID]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/BadgeID.webp

And there you go! Run the scene again, collect all the golf balls, and you should be rewarded with a badge when you're done! 🙂

![Reward]({{ sitePrefix }}/assets/img/articles/2023-09-06-How-to-create-an-Interactive-Experience-using-the-Spatial-Creator-Toolkit/End.webp

### Useful resources

* https://docs.spatial.io/
* https://docs.spatial.io/quests




_Article Photo by [Spatial](https://www.spatial.io/)_

_Spatial Unity Starter Template by [Spatial](https://docs.spatial.io/getting-started)_

How to create an Interactive Experience using the Spatial Creator Toolkit



Why split the test execution in the build process for different types of tests?

By default, all unit tests written with JUnit and Mockito will be categorised as Unit tests whereas all other types of tests
integrate one or more pieces or units of the system.

Here are for example some common context type annotations provided by SpringBoot for integration tests, which can be used
to test various components in a backend application:

```
@DataJpaTest

@DataMongoTest

@WebMvcTest

@SpringBootTest
```

If you are interested in how to set up an isolated test with SpringBoot and TestContainers,
then please refer to our [previous blog post provided with Gradle and Kotlin](https://engineering.monstar-lab.com/en/post/2023/08/01/Test-Isolation-with-TestContainers-and-SpringBoot/).

The purpose of this article is to share our own experience with a few arguments about
how to structure the common type of unit and integration tests (also shortly named as *IT) 
in Spring Boot with Maven and Java. This type of setup can be very individual 
and might be considerable only by some engineers. This knowledge share will just demonstrate how to achieve
it and how to organise them best, as well as which arguments support this idea.

#
***
#

Below are some reasons which might serve as an argument for setting up your build profiles in this way:

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/mix.webp>
<figcaption>Conflict in Intention</figcaption>
</figure>

The complexity of unit and integration tests can vary from each other by setup and the amount of components involved. 
Unit tests are normally short, they use very little dependencies or no dependencies at all and are fast in their execution. 
The Integration tests on the other hand could have the size from small to huge, 
depending on the part of the system they will be executed against and 
often instability might come into question. If the setup requires multiple dependencies which are not just Beans, 
but an entire system or containers to be provided, they will also run longer.
If a problem occurs in a unit test, it will cause the pipeline to fail. 
On the other hand, some other integration test may have already been executed before this failure, which could have been avoided
and postponed for a later phase. Parallel execution of these 2 different types of tests could alert about a failing unit test
way earlier, which means putting the shorter and simpler execution beforehand will cost less time and effort by providing early
feedback about failures.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/report.webp>
<figcaption>Code Coverage</figcaption>
</figure>

Success is measured, and in this scenario, the measurement scale is the code coverage report, 
which is normally generated and provided with specific tools such as Jacoco with SonarQube visualisation and code quality checks. 
After unit test execution, Jacoco could help collect reports from different parts of the execution in order to visualise it.
Unit tests could for example provide line coverage for the methods that they are bound to.
This is not true for the integration tests, because integration tests might aim to test or connect a few units of an application, 
but in reality, the execution can start or execute many other parts of it: as a result the report will include all parts
executed during the test, even though some units weren't intended to be part of the testing at all.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/time.webp>
<figcaption>Time is precious</figcaption>
</figure>

As unit tests are small, they also run fast. On the other hand the integration tests are not only different in size, 
but also in execution time. This can of course depend on several aspects such as how many beans need to be created, 
or which context is necessary for the application to prepare, or how many containers or systems should be started
in order to get ready for the test completion. This leads the test execution to take much longer and even 
with some intentional delays put in the code, it might prolong to some minutes.  


<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/balance.webp>
<figcaption>Stability</figcaption>
</figure>

The more "parties" are involved in the "business" the more problems and blockers are possible. 
As unit tests normally do not have many services or components as dependencies, 
they always run stable and are important to fix, because this would be the first quality control. 
Of course, it is always a good practice to achieve maximum stability for the integration tests, 
but this is not always possible or should be postponed for internal specific reasons. As many different typesIf the arguments above were convincing, then the example below  
of technologies might be used to prepare the test system, it could also be preferred to fix them at a later time.

### Technical Setup

If the arguments above were convincing, then the example below should provide a very simple technical setup
to create different build profiles which can later be used in CI/CD pipeline configuration.

In the current example a standard SpringBoot application is used with the following components

 - Controllers for endpoint calls
 - Service layer for repository access
 - Repository layer to access PostgreSQL Database


The unit tests mock all dependencies with Mockito and JUnit5 as following:

```
@ExtendWith(SpringExtension.class)
class UserServiceTest {

    @Mock
    UserRepository userRepository;

    @InjectMocks
    UserService userService;

    @Test
    void findAll() {
        //GIVEN
        User user = new User();
        user.setFirstname("TestName");
        user.setLastname("TestLastname");
        doReturn(new ArrayList<>(){{
            add(user);
        }}).when(userRepository).findAll();

        //WHEN
        List<User> users = userService.findAll();

        //THEN
        assertThat(users.get(0).getFirstname()).isEqualTo("TestName");
        verify(userRepository).findAll();
    }
}
```

In order to demonstrate the separation we also provided a full isolation test with TestContainers,
which tests the following integration in an isolated context:
Endpoint Call >> Service Layer >> Repository Layer >> PSQL Instance.


```
@Testcontainers
@SpringBootTest(classes = TestProfileApplication.class)
@ActiveProfiles("test")
@AutoConfigureMockMvc(addFilters = false)
class UserServiceIT {

  @Autowired
  UserRepository userRepository;

  @Autowired
  MockMvc mockMvc;

  @Test
  void retrieveSavedUser() throws Exception {
    //GIVEN
    var user = new User();
    user.setFirstname("Name");
    user.setLastname("Lastname");
    userRepository.save(user);

    //WHEN
    mockMvc.perform(MockMvcRequestBuilders.get("/user/{id}", 1))

    //THEN
            .andExpect((MockMvcResultMatchers.status().isOk()))
            .andExpect(result -> {
              var content = result.getResponse().getContentAsString();
              Assertions.assertThat(content).contains("Lastname");
            });
  }
}
```
For simplicity in the structure, both tests follow the famous Given-When-Then pattern for writing the tests.

And finally, to configure it, the following 2 plugin configurations for Surefire and Failsafe are added in ```pom.xml```.

```
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
                <configuration>
                    <skipTests>${skip.unit.tests}</skipTests>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
```

If we execute the usual ```mvn clean isntall``` in CLI, then both tests would execute after each other without an order.

In order to split the execution we would need to be more specific 
about which plugin we would like to make part of the maven lifecycle.
By default, in SpringBoot and Maven using the following command would skip the Integration Test
by scanning the files which end with *IT.java, 
which of course can also be configured with a different naming convention pattern if needed.

```
mvn clean -DskipITs=true verify
```

This will then let the Surefire plugin execute all tests by ignoring *IT tests.

In order to do the opposite, a flag configuration will be held for Surefire to let it ignore the unit test execution phase.
In the end, running the following command for Failsafe, will run the IT tests accordingly. 

```
mvn clean -Dskip.unit.tests=true verify
```

As a result of this setup the pipeline can be configured to run separately
to block the follow-up checks based on the results off both execution phases.

Below is a sample from GitHub Actions. 

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/pipeline-unlocked.webp>
<figcaption>Locked only by Unit test phase</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-09-01-Test-Build-Profile-for-Maven-with-Surfire-and-Failsafe/pipeline-locked.webp>
<figcaption>Locked also by Integration test phase</figcaption>
</figure>




#### Tools & References used here...
* [Monstarlab:: template with Kotlin and Gradle](https://github.com/monstar-lab-oss/TestProfile)
* [Maven as build tool](https://maven.apache.org/)
* [Java and OpenJDK as a programming and runtime environment](https://openjdk.org/)
* [Hibernate as an ORM layer for JPA](https://hibernate.org/)
* [PostgreSQL as a Datasource](https://www.postgresql.org/)
* [FlyWay for DB script migrations](https://flywaydb.org/)
* [Spring Boot for Backend Application development](https://spring.io/projects/spring-boot)
* [Test Containers for running PSQL Docker based Container](https://testcontainers.com/)
* [Unsplash Images for writing the blog post](https://unsplash.com/)
* [Brain and Mind from Monstarlab::](https://monstar-lab.com/global/)



Surefire and Failsafe: Benefits of parallel execution

Building a conversational phone callbot powered by Twilio, ChatGPT & AmiVoice


Hello there! This is Ueda from the Backend Team at MonstarLab. Recently, in a project I was involved in, the topic of using gRPC came up. So, I went through various trials and errors to quickly set up the optimal development environment. In this article, I'll introduce the method I used.

- The code is based on the Connect-go tutorial.
- We'll build a container development environment using Dev Container,
- Introduce Hot Reload using tools like Air
- Demonstrate the use of protoc-gen-validate for validation.

To focus on setting up the development environment, I'll exclude technical explanations or architecture design related to gRPC and Connect, as well as topics like CI/CD, from this article.
You can find the repository for the sample code [here](https://github.com/Yukio0315/connect-sample).

# Operating Environment

I have conducted operational tests in the following environment:

- M1 MacBook Pro (Ventura 13.4.1)
- Docker Desktop for Mac Version 4.20.1
- VSCode Version 1.79.2
- Dev Containers v0.295.0

# Technology Stack

My technology stack includes:

- [VSCode Dev Containers](https://code.visualstudio.com/docs/devcontainers/containers)
- [Golang](https://go.dev/)
- [Air](https://github.com/cosmtrek/air)
- [Connect-go](https://connect.build/docs/go/getting-started)
- [protoc-gen-validate(PGV)](https://github.com/bufbuild/protoc-gen-validate)

# Setting Up the Environment

Personally, I believe there are three key criteria for creating the optimal development environment:

1. Maintaining a certain level of code quality without relying solely on human attention.
2. Compiling written code quickly to receive immediate feedback.
3. Allowing anyone to start development promptly.

Taking these criteria into account for gRPC development environment, I have selected the following tools and libraries: VSCode Dev Container as the IDE, Golang as the programming language, Connect as the gRPC library, and libraries like Air for achieving Hot Reload. The tools chosen for this setup are considered industry standards and were also selected based on personal preferences after testing and evaluation.

# Setting Up VSCode Dev Container

I added configurations based on the [Go template](https://github.com/devcontainers/templates/tree/main/src/go) created with the "Add Dev Container Configuration Files" command from the Dev Containers extension. To install Go configurations in the container, I set up the container build using a Dockerfile. The other main configuration items are primarily these three:

- Introducing Features.
- Configuring VSCode settings and introducing extensions.
- Installing necessary tools for Go development in the Dockerfile.

While there are other things I might want to do, such as setting up git pre-commit configurations, I prioritized starting gRPC development and kept the setup minimal.

## Introducing Features

### Configuring Oh My Zsh

To establish my shell environment, I have adopted Oh My Zsh. Since people often have their own preferences when it comes to shells, it's best to choose something you like. Here are the steps to introduce Oh My Zsh to the Dev Container:

#### Installing Zsh and Oh My Zsh

By installing the [**`common-utils`**](https://github.com/devcontainers/features/pkgs/container/features%2Fcommon-utils) feature, you can incorporate both Zsh and Oh My Zsh. Configure it within the "features" section in the **`devcontainer.json`** file like this:

```json
{
  // // Features to add to the dev container. More info: https://containers.dev/features.
  "features": {
    "ghcr.io/devcontainers/features/common-utils:2": {
      "configureZshAsDefaultShell": true
    }
  }
}
```

#### Setting Zsh as the Default Shell

You can change the default shell to Zsh by adding the following script to the Dev Container's **`dockerfile`.** Reference: [stackoverflow](https://stackoverflow.com/questions/55987337/visual-studio-code-remote-containers-change-shell)

```Dockerfile
RUN echo "if [ -t 1 ]; then" >> /root/.bashrc
RUN echo "exec zsh" >> /root/.bashrc
RUN echo "fi" >> /root/.bashrc
```

#### Configuring Oh My Zsh in .zshrc

Include the configuration for Oh My Zsh in the **`.zshrc`** file within the **`.devcontainer`** directory. You can use the [**`zsh-template`**](https://github.com/ohmyzsh/ohmyzsh/blob/master/templates/zshrc.zsh-template) from Oh My Zsh for this purpose.

#### Copying .zshrc to the Container

Finally, add the following line to your Dockerfile to copy the **`.zshrc`** file to the home directory within the container:

```Dockerfile
ADD .zshrc $HOME
```

## Configuring VSCode Settings and Introducing Extensions

Now let's dive into configuring VSCode. We'll start with the **`settings.json`** configuration, where I will use the official [default settings](https://code.visualstudio.com/docs/getstarted/settings#_default-settings). You can customize these settings to match your personal preferences. When it comes to introducing extensions, this is the part where personal preferences shine through the most. In my case, I went with the following setup. Since these are all well-known extensions, I'll skip detailed descriptions for each. Just remember to add the necessary configurations to your settings files as needed.

In your **`.devcontainer.json`**:

```json
{
  "customizations": {
    "vscode": {
      "extensions": [
        "EditorConfig.EditorConfig",
        "streetsidesoftware.code-spell-checker",
        "wayou.vscode-todo-highlight",
        "ms-azuretools.vscode-docker",
        "shardulm94.trailing-spaces",
        "bungcip.better-toml"
      ]
    }
  }
}
```

## Configuring Dockerfile for Golang Development

An essential part of Golang development is the [Go extension](https://github.com/golang/vscode-go) from Extensions. Installing this extension is enough to get started with development.

In your **`.devcontainer.json`**:

```json
{
  // Configure tool-specific properties.
  "customizations": {
    "vscode": {
      "extensions": ["golang.go"]
    }
  }
}
```

In the Dockerfile, you'll want to install dependencies required by the Go extension, such as gopls and dlv, along with any other necessary tools to complete your Golang development environment.

`Dockerfile`

```Dockerfile
RUN go install github.com/cweill/gotests/gotests@latest
RUN go install github.com/josharian/impl@latest
RUN go install github.com/go-delve/delve/cmd/dlv@latest
RUN go install honnef.co/go/tools/cmd/staticcheck@latest
RUN go install golang.org/x/tools/gopls@latest
```

# Setting Up Connect-go Environment

With the Golang environment set up as described in the previous sections, we can now proceed to set up Connect-go. Essentially, by following the [tutorial](https://connect.build/docs/go/getting-started/), you can easily get Connect-go up and running.

## Introducing Extensions

For the **Dockerfile**, add the following commands based on the tutorial's instructions:

`Dockerfile`

```
RUN go install github.com/bufbuild/buf/cmd/buf@latest
RUN go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest
RUN go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
RUN go install github.com/bufbuild/connect-go/cmd/protoc-gen-connect-go@latest
```

## Writing Sample Code

Following the tutorial, create **`go.mod`**, **`buf.yaml`**, and **`buf.gen.yaml`** files, write your main code, and execute **`main.go`** to complete the Connect-go setup.

## Introducing Hot Reload

To enhance the development environment, we will introduce Hot Reload. Hot Reload requires two steps:

1. Automatic code generation.
2. Hot Reload for Golang code.

### Automatic Code Generation

For automatic code generation, utilize the **`buf generate`** command upon saving proto files. You can achieve this using the "Run on Save" [extension](https://github.com/emeraldwalk/vscode-runonsave). Add **`"emeraldwalk.RunOnSave"`** to your **`devcontainer.json`** and configure it in **`.vscode/settings.json`**:

```json
{
  // Run on Save
  // proto-gen
  "emeraldwalk.runonsave": {
    "commands": [
      {
        "match": "\\.proto$",
        "cmd": "rm -rf gen | buf generate"
      }
    ]
  }
}
```

This **`cmd`** is used to remove all files within the **`gen`** directory. This precaution is taken because the **`buf generate`** command might not remove inadvertently generated files.

As the scale of your project increases, I think I will need to reconsider it and potentially implement a more refined approach.

### Introducing Golang's Hot Reload with Air

For Golang's Hot Reload, we've introduced the use of [Air](https://github.com/cosmtrek/air). There are various ways to install it, but by adding **`RUN go install github.com/cosmtrek/air@latest`** to your Dockerfile, the setup is complete. The configuration file needs to be set up as well. Running the **`air init`** command within the container will generate the **`.air.toml`** file, which you can modify based on the [**`.air_sample.toml`**](https://github.com/cosmtrek/air/blob/master/air_example.toml), adapting it for your project's structure.

In this setup, we've set **`delay=500`**. _This delay is important_ because without it, when **`RunOnSave`** operates, it might not be able to read files under the **`gen`** directory.

Once the Hot Reload configuration is in place, make sure Air automatically starts after the container is up. Use the **`postStartCommand`** in your **`devcontainer.json`**:

```json
{
  "postStartCommand": "air -c .air.toml"
}
```

With these steps, your Hot Reload configuration is now fully set up.

## Adding Request Validation Feature

Now, let's move on to adding the request validation feature, something that would be useful in practical usage. We'll use [protoc-gen-validate](https://github.com/bufbuild/protoc-gen-validate) (PGV) for Connect's validation.

Add **`RUN go install github.com/envoyproxy/protoc-gen-validate@latest`** to your Dockerfile. Then, as documented, add the necessary configurations to **`buf.gen.yaml`** and **`buf.yaml`**. Be careful not to forget adding **`opt`** according to your environment's needs.

`buf.gen.yaml`

```yaml
plugins:
  - plugin: buf.build/bufbuild/validate-go
    out: gen
    opt: paths=source_relative
```

`buf.yaml`

```yaml
deps:
  - buf.build/envoyproxy/protoc-gen-validate
```

After this, run the **`buf mod update`** command to generate the **`buf.gen.yaml`** file. Be aware that automatically running **`bef generate`** might prevent you from receiving error messages. If things don't work smoothly, manually running the commands is recommended.

Now, let's implement validation. Import **`validate`** in your **`greet.proto`** file and modify the validation rules for **`GreetRequest`**, like so:

`greet/v1/greet.proto`

```proto
syntax = "proto3";

package greet.v1;

import "validate/validate.proto";

option go_package= "connect-sample/gen/greet/v1;greetv1";

message GreetRequest {
  string name = 1 [(validate.rules).string = {min_len: 5, max_len: 10}];
}

message GreetResponse {
  string greeting = 1;
}

service GreetService {
  rpc Greet(GreetRequest) returns (GreetResponse) {}
}

```

Running the **`buf generate`** command will generate the **`gen.pb.validate.go`** file. Upon inspecting its contents, you'll notice that the **`GreetRequest`** type struct includes methods like **`Validate`** and **`ValidateAll`**. You can use these methods. For instance, you can add validation like this:

```go
func (s *GreetServer) Greet(
 ctx context.Context,
 req *connect.Request[greetv1.GreetRequest],
) (*connect.Response[greetv1.GreetResponse], error) {
	if err := req.Msg.Validate(); err != nil {
	  return nil, err
	}
	log.Println("Request headers: ", req.Header())
	res := connect.NewResponse(&greetv1.GreetResponse{
		Greeting: fmt.Sprintf("Hello, %s!", req.Msg.Name),
	})
	res.Header().Set("Greet-Version", "v1")
	return res, nil
}

```

With this, the validation setup is complete.

# Testing the Setup

You can test the setup using the following three commands:

```shell
git clone https://github.com/Yukio0315/connect-sample

devcontainer open # if devcontainer CLI installed

root ➜ /workspaces/connect-sample (main) $ go run ./cmd/client/main.go
YYYY/MM/DD hh:mm:ss unknown: invalid GreetRequest.Name: value length must be between 5 and 10 runes, inclusive
```

With these steps, you should see that the server runs without issues and the validation is working as expected.

# Conclusion

This article has explained how to set up the optimal gRPC development environment using Connect-go within a Dev Container, all in a speedy manner. By incorporating features like linting and hot reload, I have been able to create an environment that's fairly ideal for development. While there might still be areas that need improvement, I have managed to establish the essential environment needed to kickstart development. I hope this article proves helpful to those involved in gRPC development and beyond.

_Article Photo by [Simon Berger](https://unsplash.com/photos/aZjw7xI3QAA)_


Setting Up the Optimal gRPC Development Environment Using Connect-go and VSCode Dev Container


SwiftUI was initially announced 4 years ago, and last year Apple said clearly that the best way to build an app is with Swift and SwiftUI. However, some of us are still working on codebases that were started in UIKit more than 4-5 years ago and that is possibly holding us back from adopting SwiftUI. But as long as you target iOS 13 or above, there are ways to use SwiftUI even if you have a UIKit project. The way to do it depends on what exactly you want to build in SwiftUI: is it a full screen? A view to be shown inside a UIKit screen? Or do you have a table or collection view cell that you want to build in SwiftUI?

![]({{ sitePrefix }}/assets/img/articles/2023-07-24-SwiftUI-in-UIKit/best.webp)
_Capture from WWDC 2022 Session 102 Platforms State of the Union, 29:20_

## Whole SwiftUI screen

Let's say you have an existing app, where you need to build a new feature that is its own separate screen. That's a perfect candidate to do it using SwiftUI. For that, we use a `UIHostingController`. So first, we build the new screen in SwiftUI. Here's a very simple example of a static screen:

```swift
import SwiftUI

struct SwiftUIScreen: View {
    var body: some View {
        VStack(spacing: 24) {
            Text("Hello, from SwiftUI!")
            Image(systemName: "globe")
        }
    }
}
```

This gives the following view when shown on the whole screen:
![]({{ sitePrefix }}/assets/img/articles/2023-07-24-SwiftUI-in-UIKit/1.webp)

Then we need to push or present or show in whatever way the SwiftUI screen that we just built, from UIKit, using a `UIHostingController`. In our case, I have a very simple `UIViewController`, in which I have added a `UIButton` which presents this screen modally.

```swift
import UIKit
import SwiftUI

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view.
    }

    @IBAction func openSwiftUIScreen(_ sender: Any) {
        let vc = UIHostingController(rootView: SwiftUIScreen())
        present(vc, animated: true)
    }
}
```

That was one simple way to integrate a SwiftUI screen in a UIKit app. 

Of course, in a real-life project, things are not as simple as here, we normally have more dependencies that we need to inject, possibly a ViewModel depending on the choice of architecture. However, the basics are the same: in your UIKit code, instantiate your SwiftUI screen, embed it into a `UIHostsingController` (which inherits from `UIViewController`) and then present or push or do what needs to be done with that hosting controller.

## Smaller SwiftUI view

But what if we want to only build a small view in SwiftUI and integrate it in a `UIViewController`, as opposed to a whole SwiftUI screen? We can use the same technique, with `UIHostingController`, because a `UIViewController` doesn't have to cover the whole screen. So what we're doing basically is adding a child view controller to host the SwiftUI view. Here's how we can do that.

Here I have `MixedViewController` which is a `UIViewController`, in which I add a `UIStackView` that covers the whole screen. In the stack view, I add 2 arranged subviews: a `UILabel`, and the same SwiftUI View that we used above (`SwiftUIScreen`).

```swift
import UIKit
import SwiftUI

class MixedViewController: UIViewController {
    
    private let stackView = UIStackView()
    private let label = UILabel()
    
    override func viewDidLoad() {
        super.viewDidLoad()

        view.backgroundColor = .systemBackground
        setupStackView()

        label.text = "This is a UILabel"
        label.textAlignment = .center
        stackView.addArrangedSubview(label)
        
        let swiftUIController = UIHostingController(rootView: SwiftUIScreen())
        addChild(swiftUIController)
        stackView.addArrangedSubview(swiftUIController.view)
        swiftUIController.didMove(toParent: self)
    }
    
    private func setupStackView() {
        stackView.axis = .vertical
        stackView.distribution = .fillEqually
        
        stackView.translatesAutoresizingMaskIntoConstraints = false
        view.addSubview(stackView)
        NSLayoutConstraint.activate([
            stackView.topAnchor.constraint(equalTo: view.topAnchor),
            stackView.leadingAnchor.constraint(equalTo: view.leadingAnchor),
            stackView.bottomAnchor.constraint(equalTo: view.bottomAnchor),
            stackView.rightAnchor.constraint(equalTo: view.rightAnchor),
        ])
    }
}
```

As you can see in the code above, the way to embed a SwiftUI view in a `UIViewController` is very similar to embedding a child view controller in a `UIViewController`. In fact, it's exactly the same, because we embed the SwiftUI view in a `UIHostingController`, which inherits from `UIViewController`. 
Here's what our `MixedViewController` looks like when running the app.
![]({{ sitePrefix }}/assets/img/articles/2023-07-24-SwiftUI-in-UIKit/2.webp)

## SwiftUI in table or collection cells

If you target iOS 16 or above, there's also another way of adding SwiftUI in a UITableViewCell or UICollectionViewCell.
Apple introduced `UIHostingConfiguration`, which allows developers to declare the contents of a UITableViewCell or UICollectionViewCell inline, using SwiftUI. Here's a simple example to show this:

```swift
import UIKit
import SwiftUI

class TableViewController: UITableViewController {

    override func viewDidLoad() {
        super.viewDidLoad()

        tableView.register(UITableViewCell.self, forCellReuseIdentifier: "reuseIdentifier")
    }

    // MARK: - Table view data source

    override func numberOfSections(in tableView: UITableView) -> Int {
        return 1
    }

    override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        return 4
    }
    
    override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        let cell = tableView.dequeueReusableCell(withIdentifier: "reuseIdentifier", for: indexPath)

        cell.contentConfiguration = UIHostingConfiguration {
            HStack {
                Text("SwiftUI cell - Row \(indexPath.row)")
                Spacer()
                Image(systemName: "globe")
            }
        }

        return cell
    }
    
}
```

And here's the screen with a `UITableView` with SwiftUI content in the `UITableViewCell`s if we run it:

![]({{ sitePrefix }}/assets/img/articles/2023-07-24-SwiftUI-in-UIKit/3.webp)

And those are three ways to use SwiftUI in UIKit: either use a `UIHostingController` initialised with a SwiftUI root view and show that full screen or as a child view controller, or if you can target iOS 16 or above, use a `UIHostingConfiguration` to add collection view or table view cells content inline using SwiftUI.

Please note that all the code examples in this blog post are meant to be as concise and to the point as possible, illustrating how to embed SwiftUI content in UIKit. Don't use it as best practice guidelines for anything else because the main and only point they have was to show how to use SwiftUI in UIKit.

And just because I'm showing how to use SwiftUI in a UIKit project doesn't mean UIKit is obsolete. For example, it's [still a thing in visionOS](https://developer.apple.com/videos/play/wwdc2023/111215/).

_Article photo by <a href="https://unsplash.com/@varpap">Vardan Papikyan</a> on <a href="https://unsplash.com/photos/JzE1dHEaAew">Unsplash</a>_
   

How to use SwiftUI in a UIKit app

Empowering Mobile Applications through Blockchain: Our Journey, Part 2



Are you using SpringBoot for backend services and looking to integrate TestContainers into your testing setup? 

If so, then this example should give you an idea of how to achieve it in the shortest amount of time using the least possible configuration to set the implementation up. 
To do this we will be considering Isolated Testing as a use case and use the TestContainers 
to achieve this in the most simple way possible in order to run an integration test.
This setup would be more suitable for the projects where integration tests have higher priority and are part of regular execution in the CI/CD pipeline.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-08-01-Test-Isolation-with-TestContainers-and-SpringBoot/test.webp>
<figcaption>Testing Lab</figcaption>
</figure>

The purpose of isolated test, with or without TestContainers, is very important in Backend applications, 
because at some point the integration tests should be stable enough to make part of regular execution in CI/CD pipeline chain.

#
>***Besides, we want to make sure tests never interfere with each other during the execution and especially the technical 
>setup does not enforce more and more refactoring over the time because of dependencies among the tests.***
#

#
To ensure the correctness of the test results and the stability of each test method execution,
we need to make sure the internal or external services used in the integration test context
do not produce a data collision, and each test method execution can set up the data state for the service it requires.
This could be done for example on an infrastructure premises by running separate instances of services on which our tests would depend.
By doing so, we could achieve the goal, but not in an efficient way.
This for example would require more effort to set it up and also to make sure
that the test results are not impacting each other before, after and during the execution.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-08-01-Test-Isolation-with-TestContainers-and-SpringBoot/result.webp>
<figcaption>What could possibly go wrong?</figcaption>
</figure>

What problems would we encounter with this setup? The earlier will be too expensive and 
the later one will is not flexible enough to prevent parallel test execution.

Of course, we could make sure the data is clean in the datasource before starting the tests to support parallel execution, 
but the cost of knowing the whole test system in order to modify or extend it would be 
a heavy task and never safe because of the risk of producing bugs within the test code.

Luckily, TestContainers would make this easy for us so that we do not need to worry
about the data collision or parallel execution.
Why? Depending on our setup, the technology would simply take care of it 
by creating and destroying the type of instance we require for our integration test either for each test case or test class. 
The execution in this manner might be a little more expensive, 
but the cost pays us with some values of easy setup, stability, flexibility and independence in tests.
Besides, making sure that both old and new tests written at 
any point in time are still independent of each other could save lots of refactoring time.



<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-08-01-Test-Isolation-with-TestContainers-and-SpringBoot/design.webp>
<figcaption>How to design an integration test?</figcaption>
</figure>

With new releases of SpringBoot and TestContainers it is way easier to accomplish the integration. 
Depending on the use case, as in unit testing, as well as in the integration test, 
the recommended structure to use in the test code is the classical testing strategy based on GIVEN/WHEN/THEN concept as following:

>
> 1. Given: Data Preparation: Prepares or mocks the test data. 
> . 
> 2. When: Target Code: Executes the testable production code.
> 
> 3. Then: Assertion: Checks the expected data.

#

What is different here compared to unit testing is that we not only need to prepare 
the test data but also, depending on the requirements, provide external and/or internal 
services for the test setup, allowing each test case to manage them individually according to their needs. 
For example, if our integration test depends on a DB instance, then we would need to provide 
a Docker based DB instance before the test execution and insert the necessary data.

#
***
#

```
@RestController
@RequiredArgsConstructor
internal class UserController(val service: UserService) {

    @GetMapping("/users")
    fun all(): Iterable<User> {
        return service.findAll()
    }

    @PostMapping("/users")
    fun newUser(@RequestBody newUser: User): User {
        return service.createNew(newUser)
    }

    @GetMapping("/users/{id}")
    fun one(@PathVariable id: Long): User? {
        return service.getUser(id) ?: throw UserNotFoundException(id)
    }
}
```

This specific example has been developed as a basic SpringBoot application
which exposes endpoints to insert and fetch basic data from PostgreSQL datasource.
If we were to write multiple integration tests for this application we could either
run multiple PSQL instances by creating and destroying them per test case or class or
use 1 instance and prepare the data state before and after each execution.

```
@Testcontainers
@SpringBootTest(classes = [IsolatedTestingApplication::class])
@ActiveProfiles("test")
@AutoConfigureMockMvc(addFilters = false)
class UserApiIT {

    @Autowired
    var userRepository: UserRepository? = null

    @Autowired
    var mockMvc: MockMvc? = null

    @Test
    fun `Retrieve User from DB via the Endpoint`() {
        //GIVEN        
        val user = User("Name", "Lastname")
        userRepository?.save(user)

        //WHEN
        mockMvc!!.perform(
            MockMvcRequestBuilders.get("/users/{id}", 1))
            
        //THEN
                .andExpect(MockMvcResultMatchers.status().isOk)
                .andExpect { result: MvcResult ->
                    val content = result.response.contentAsString
                    assertThat(content).contains("Lastname")}
                .andReturn())
    }
}
```


In the test case, the requests will be made by using Spring Mock MVC to reach the PSQL Datasource, which we can then set up by using
Spring Boot configuration file and TestContainers provided PostgreSQL Docker based instance.

Next, the configuration would make sure that the SpringBoot and TestContainer integration could set up and run a test application before the test case execution: 

appication.yml

```
spring:
  datasource:
    url: jdbc:postgresql://${DATABASE_HOST:localhost}: ${DATABASE_PORT:5435}/${DATABASE_NAME:template}
    username: ${DATABASE_USER:templateUser}
    password: ${DATABASE_PASSWORD:templatePassword}
```

appication-test.yml

```
spring:
  datasource:
    url: jdbc:tc:postgresql:14:///template
```
This will automatically initiate an PSQL Docker based instance by creating the mentioned user as well as the database in it.
As soon as the Docker instance is ready SpringBoot application startup will be triggered along with the FlyWay migration scripts to be executed.
Based on the given configuration the Docker PSQL instance will have the following setup ready for the application to connect and use...

 * Host: <i>localhost</i> 
 * Username: <i>templateUser</i>
 * Password: <i>templatePassword</i>
 * Database: <i>template</i>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-08-01-Test-Isolation-with-TestContainers-and-SpringBoot/docker.webp>
<figcaption>Docker and Containers in action</figcaption>
</figure>

While running the provided test sample you can simply follow in the Docker Desktop that 2 Docker Containers are appearing and disappearing on the fly to support our test execution on. 

#
***
#

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-08-01-Test-Isolation-with-TestContainers-and-SpringBoot/kts.webp>
<figcaption>
Technical Setup
</figcaption>
</figure>

The source code for the described use case is available in [this GitHub repository](https://github.com/monstar-lab-oss/Isolated-Testing) provided by [Monstarlab::](https://monstar-lab.com/global/)
Having [Docker Desktop](https://www.docker.com/) installed is a pre-requisite to run it. 


#
***
#

Executing the following command in CLI should be sufficient to see and check the test results.

```agsl
./gradlew clean build
```

#
***
#

The following steps are necessary to run the backend application in a CLI:

1. Initiate PSQL Docker Container with a Database. 
``` 
docker run -d --name template_postgres -e POSTGRES_USER=templateUser -e POSTGRES_PASSWORD=templatePassword -e POSTGRES_DB=template -p 5435:5432  --restart=always postgres
```

2. Start the backend application
```
./gradlew bootRun
```


The choice of Kotlin as a language and Gradle as a build tool is merely one step forward towards the modern programming world and has no impact on the setup:
this could have been easily also achieved with Maven and Java.


#### Tools & References used here...
* [Monstarlab:: template with Kotlin and Gradle](https://github.com/monstar-lab-oss/spring-boot-template)
* [Gradle as build tool with KTS](https://gradle.org/)
* [Kotlin as a programming language](https://kotlinlang.org/)
* [Hibernate as an ORM layer for JPA](https://hibernate.org/)
* [PostgreSQL as a Datasource](https://www.postgresql.org/)
* [FlyWay for DB script migrations](https://flywaydb.org/)
* [Spring Boot, Note that since 3.1 TestContainers are supported by SrpingBoot](https://spring.io/projects/spring-boot)
* [Test Containers for running PSQL Docker based Container](https://testcontainers.com/)
* [Unsplash Images for writing the blog post](https://unsplash.com/)
* [Brain and Mind from Monstarlab::](https://monstar-lab.com/global/)



Isolated Testing with Test Containers and Spring Boot in Kotlin


Welcome back! In this blog post, we will explore Appium, a powerful tool for automating Hybrid Android apps. Firstly, we will discuss locating Hybrid app elements in Appium, which involves identifying and interacting with specific elements within the app's user interface. Next, we will cover Hybrid Android app automation, which focuses on automating the testing process of Android apps with hybrid features. Additionally, we will explore the Page Object Model (POM) pattern for Appium test automation, a structured approach that aids in maintaining maintainable and reusable code. Finally, we will highlight best practices for Appium test automation, providing valuable guidelines for achieving reliable and efficient automation. 

By the end of this blog, you will have the knowledge and tools to enhance your Hybrid Android app automation process using Appium. If you haven't read the [first part](https://engineering.monstar-lab.com/en/post/2023/04/26/Accelerate-Your-Android-App-Testing-with-Appium-and-Java-A-Beginner%E2%80%99s-Guide-(Part-1)/) yet, I highly recommend checking it out before proceeding. In the first part of this blog post series, we explored:
 - How to set up Appium for Android test automation
 - Getting the appPackage and appActivity
 - Writing your first Appium test

Now let’s deep dive. Hybrid Android app refers to an application that combines elements of both native and web-based technologies for development. It typically uses web technologies such as HTML, CSS, and JavaScript to build the user interface (UI) and runs inside a WebView component, which acts as a browser embedded within the app.

## 1. Locating Hybrid app elements in Appium

Hybrid apps are built using a combination of web technologies (HTML, CSS, JavaScript) and native components. In that case, you need to locate both web and native elements.

### Locating Web Elements: 

Hybrid apps typically use a WebView component to display web content within the app. You'll need to locate the WebView element to interact with the web elements inside it. You can locate Web elements following these steps:

1. Install the hybrid app on an Android device or emulator. If you're running on it on a real device, connect the Android device to your development machine using a USB cable. Make sure USB debugging is enabled in the device's developer options. Ensure that the app is running in a debuggable mode.

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/USBDebuuging.webp)

2. Open Google Chrome on your development machine and navigate to "chrome://inspect". You should see a list of connected devices and running WebView instances. 

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/ChromeInspector.webp)

3. Find the WebView instance corresponding to your app and click the "Inspect" button. This opens the Chrome DevTools for that WebView.

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/ChromeInspecter2.webp)

4. Within Chrome DevTools, you can use various inspection tools like the Elements panel, Console, and Network panel to locate and interact with web elements. You can inspect the HTML structure, apply CSS selectors, execute JavaScript, and simulate user interactions.


5. Once you have identified the desired web element, you can use it in your code to interact with the element.


### Locating Native Elements:

When it comes to locating native elements in Appium, there are several ways to do so. One popular method is to use Appium Inspector, which is a visual tool that allows you to inspect the user interface (UI) elements of your application and generate the corresponding locators.
To use Appium Inspector, you'll need to have your app running on a device or emulator and have the Appium server running. Once you have both of these setups, you can locate elements following these steps:

1. Download and Install the Appium Inspector from [here](https://github.com/appium/appium-inspector/releases).



2. Open Appium Inspector and Enter the desired capabilities for your test, such as the device name and platform name. Find the details documentation of desired capabilities from [here](https://appium.readthedocs.io/en/stable/en/writing-running-appium/caps/).

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/AppiumInspecter.webp)

3. Click the "Start Session" button to launch your app on the device or emulator.



4. Once your app is running, you should see the UI elements displayed in Appium Inspector. You can click on the UI elements and see their attributes, such as the ID, class name, and text.

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/AppiumInspecter2.webp)

5. To generate a locator for an element, simply click on the element in Appium Inspector and Copy "XPath" / "Accessibility ID" or “Resource ID” (depending on your preferred locator type) from the right side of the Selected Element.


6. Paste the generated locator into your Appium test code and use it to interact with the element.


Using Appium Inspector you can save you time and make it easier to locate elements in your app. However, it's important to note that it's not always the most reliable method, especially if the UI of your app changes frequently. In these cases, you may need to use alternative methods such as [regular expressions](https://www.lambdatest.com/blog/locators-in-appium/) to locate your elements.

## 2. Hybrid Android app Automation with Appium

Automating Hybrid apps can be more complex compared to testing native apps or web apps alone, as you need to handle both web and native contexts in your test script. Fortunately, Appium provides methods to switch between these contexts and perform actions accordingly.

Here's an example code snippet to give you an idea of how to automate a hybrid Android app using Appium with Java:

Switching into Webview:

```java
    public static void switchToWebView() {
        Set<String> contextHandles = driver.getContextHandles();
        for (String context : contextHandles) {
            if (context.startsWith("WEBVIEW")) {
                driver.context(context);
                break;
            }
        }
    }
```
Switching into Nativeview:
```java

    public static void switchToNativeView() {
        Set<String> contextHandles = driver.getContextHandles();
        for (String context : contextHandles) {
            if (context.startsWith("NATIVE")) {
                driver.context(context);
                break;
            }
        }
    }

```

The given code provides two methods, `switchToWebView()` and `switchToNativeView()`, for switching between the web view and native view in a mobile application using Appium.

In the `switchToWebView()` method:

- The code starts by obtaining a set of all available context handles using the `driver.getContextHandles()` method. Context handles represent different execution contexts or views within the application.
- It then iterates through each context handle using a for-each loop.
- Inside the loop, it checks if the context handle starts with the prefix "WEBVIEW" using the startsWith() method.
- If a context handle with the "WEBVIEW" prefix is found, it indicates the web view context.
- The code then switches the driver's context to the identified web view context using the driver.context(context) method.
- Finally, the loop is broken using the break statement to prevent unnecessary iterations once the web view context is found.

Similarly, in the `switchToNativeView()` method, the code performs the following steps:

- Obtains the set of all available context handles using driver.getContextHandles().
- Iterates through each context handle.
- Checks if the context handle starts with the prefix "NATIVE".
- If a context handle with the "NATIVE" prefix is found, it indicates the native view context.
- Switches the driver's context to the identified native view context using `driver.context(context)`.
- Breaks the loop to stop further iterations once the native view context is found.

These methods provide a way to switch between the web view and native view contexts in a mobile application when using Appium. There are some other methods such as `driver.getContextHandles()`, `driver.getContext()`, `driver.switchTo().context()`
are used to get context and switching the context between Web and Native view.  


## 3. POM for Appium test Automation

POM stands for Page Object Model, which is a design pattern used in test automation to represent each page of an application as a separate class, containing its own set of methods and properties. By separating the application pages into separate classes, the code becomes more readable and easier to understand.

![]({{ sitePrefix }}/assets/img/articles/2023-07-14-Accelerate-Your-Android-App-Testing-with-Appium-and-Java-(Part-2)-Hybrid-App-Automation/POM.webp)

### Step 1: Create Page Objects

The first step is to create a class for each page or screen of the application that you want to test. The class should contain the elements and methods specific to that page. For example, if you have a login page, you can create a LoginPage class as follows:

```java
public class LoginPage {
   private AndroidDriver<AndroidElement> driver;
   public LoginPage(AndroidDriver<AndroidElement> driver) {
      this.driver = driver;
   }
   private By emailId = By.id(“email”);
   private By password = By.id(“password”);
   private By loginButton = By.id(“login”);

   public void enterEmail(String email) {
      driver.findElement(emailId).sendKeys(email);
   }
   public void enterPassword(String pass) {
      driver.findElement(password).sendKeys(pass);
   }
   public void clickLoginButton() {
      driver.findElement(loginButton).click();
       // Check if the login is successful 
      if (/* condition to check if login is successful */) { 
        // Success assertion Assert.assertTrue
       (/* condition indicating login success */, "Login was successful."); 
       returnToHome();
       } 
     else { 
       // Fail assertion Assert.fail
      ("Login failed."); 
      }
  }
 private void returnToHome() { 
    // Code to navigate back to the home screen or perform other actions 
}
}
```

### Step 2: Create a TestCases Class

Creates a separate class for each test case that you want to execute. In each test class, initialize the driver and create an instance of the page class. For example, if you want to test the login functionality, you can create a test class as follows:

```java

public class LoginTest {
   private AndroidDriver<AndroidElement> driver;
   LoginPage loginPage;
   @BeforeTest
   public void setUp() throws MalformedURLException {
      DesiredCapabilities caps = new DesiredCapabilities();
      caps.setCapability(“deviceName”, “emulator-5554");
      caps.setCapability(“platformVersion”, “10.0");
      caps.setCapability(“deviceName”, “emulator-5554");
      caps.setCapability(“appPackage”, “com.example.android”);
      caps.setCapability(“appActivity”, “.MainActivity”);
      caps.setCapability(“automationName”, “UiAutomator2");
      URL url = new URL(“http://127.0.0.1:4723/wd/hub”);
      driver = new AndroidDriver<AndroidElement>(url, caps);
      loginPage = new LoginPage(driver);
   }
   @Test
   public void testLogin() {
      loginPage.enterEmail(“testuser@test.com”);
      loginPage.enterPassword(“testpassword”);
      loginPage.clickLoginButton();
   }
   @AfterTest
   public void tearDown() {
      driver.quit();
   }
}
```

### Step 3: Run the Test

Run all the test classes and verify the results. You can use a test runner such as TestNG or JUnit to run the tests. Find the demo project for POM [here](https://github.com/Eftiar/DemoPageObjectModel). 

### 4: Best practices for Appium test automation



- **Use stable locators**: Choose reliable and unique locators to identify elements within the app's UI. IDs or names are better to use for web elements, while resource IDs or accessibility IDs are useful for native elements. Avoid relying solely on element positions, as they can change dynamically.

- **Set up the desired capabilities correctly:** Provide accurate and complete desired capabilities to establish the correct session for your hybrid app. Specify the app type and include other necessary capabilities like platform name, device name, app package, app activity, and automation name.

- **Use waits and synchronization:** Hybrid apps often involve asynchronous operations, such as loading web content or making API requests. Use implicit and explicit waits to handle synchronization. Implicit waits set a global timeout for element searches, while explicit waits wait for specific conditions to be met before proceeding with the test. Learn more about waits from here. Here’s an example of how to use waits:

    ```java
    public class HybridAppTest { 

    private WebDriver driver; 
    private WebDriverWait wait; 
    public HybridAppTest(WebDriver driver) { 
    this.driver = driver; 
    this.wait = new WebDriverWait(driver, 10); 
    // Initialize WebDriverWait with a timeout of 10 seconds 
    } 
    public void performHybridAppTest() { 
    // Perform some actions that trigger the loading of web content 
    // ... 
    // Wait for a specific element to be visible using explicit wait 
    By elementLocator = By.id("exampleElementId"); wait.until(ExpectedConditions.visibilityOfElementLocated(elementLocator)); 
    // Perform actions on the loaded element 
    WebElement element = driver.findElement(elementLocator); element.click(); 
    // Continue with the test 
    // ... 
    } 
    }
    ```

- **Use parameterization:** Parameterizing your tests can help you to run the same test with different data inputs, making your tests more flexible and reusable. You can use tools such as TestNG or JUnit to define data-driven tests. Here’s an example of how to use parameterization:

    ```java
    public class ParameterizedTest { 

    @DataProvider(name = "testdata") 
    public Object[][] testData() { 
    return new Object[][] { 
    {"username1", "password1"}, 
    {"username2", "password2"}, 
    {"username3", "password3"}
        };
    } 
    @Test(dataProvider = "testdata") 
    public void loginTest(String username, String password) { 
    // Perform login with the provided username and password 
    // ... 
    }
    }
    ```

- **Continuous integration and reporting:** Integrate your hybrid app automation tests into a continuous integration (CI) system, such as Jenkins or CircleCI. This allows for automatic test execution on code changes and provides detailed test reports and notifications. Additionally, leverage reporting tools like Extent Reports or Allure to generate comprehensive and visually appealing test reports.

 - **Use cloud-based devices:** Testing on real devices can be expensive and time-consuming. Using a cloud-based device provider such as AWS Device Farm, Sauce Labs, or BrowserStack can help you test your app on a wide range of devices and configurations without managing physical devices.


In conclusion, this two-part blog post series provides valuable insights into accelerating Android app testing with Appium and Java, specifically focusing on hybrid app automation. The first part covers the initial setup of Appium, obtaining the necessary package and activity information, and writing your first Appium test. The second part delves deeper into Appium by discussing best practices for locating hybrid app elements, automating hybrid Android apps, implementing the Page Object Model (POM) for test automation, and highlighting general best practices for Appium test automation.


_Article Photo by [Alex Knight](https://unsplash.com/@agk42)_








Accelerate Your Android App Testing with Appium and Java (Part 2)- Hybrid App Automation


In today's fast-paced world, businesses need to quickly develop and deploy web and mobile applications to stay competitive. AWS Amplify, a powerful development framework provided by Amazon Web Services (AWS), enables developers to build scalable and secure applications with ease. In this article, we'll explore AWS Amplify and delve into its key features and benefits.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/1amplify.webp>
<figcaption>What is Amplify?</figcaption>
</figure>

## What is AWS Amplify?

AWS Amplify is a comprehensive set of tools and services designed to simplify the development and deployment of web and mobile applications. It offers a seamless workflow for frontend and backend development, including features such as authentication, storage, APIs, and serverless functions.

## Key Features of AWS Amplify

1. **Authentication**: Amplify provides built-in authentication capabilities, including authentication providers such as Amazon Cognito, social sign-ins, and even custom authentication flows.
    Here is an example:

First, add the authentication library:

```commandline
amplify add auth
```
In your React application directory, install the Amplify library:
```commandline
npm install aws-amplify @aws-amplify/ui-react
```
In your, _App.js_ make sure you import the required components you need from the _aws-amplify/ui-react_ library. For now, we are importing _withAuthenticator_ and _AmplifyAuthenticator_:

```javascript
import React from 'react';
import { withAuthenticator, AmplifySignOut } from '@aws-amplify/ui-react';
const App = () => ( <div> <AmplifySignOut /> My App </div> ); export default withAuthenticator(App);
```
You will also notice the _AmplifySignOut_ component here, which renders a signout button.

2. **Data Storage**: With Amplify, developers can easily integrate data storage solutions such as Amazon DynamoDB, Amazon S3, or even custom data sources to store and retrieve application data. Here’s an example:

The fastest way to get started with using datastore in React is:
```commandline
npx create-react-app amplify-datastore --use-npm
cd amplify-datastore
npx amplify-app@latest
```
3. **APIs and AppSync**: Amplify simplifies the process of creating and integrating APIs into your applications. It supports both REST and GraphQL APIs and provides a robust backend infrastructure with features like real-time data synchronization using AWS AppSync.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/2api.webp>
<figcaption>Amplify's API</figcaption>
</figure>

4. **Serverless Functions**: Amplify integrates with AWS Lambda to enable developers to build and deploy serverless functions easily. This allows for the execution of backend logic without worrying about managing servers or infrastructure.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/3serverless.webp>
<figcaption>Serverles functions</figcaption>
</figure>

5. **Hosting and Continuous Deployment**: Amplify offers a streamlined process for hosting and deploying web applications. It supports automatic deployment through Git-based workflows, making it easy to deploy updates to your application in a controlled and efficient manner.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/4cicd.webp>
<figcaption>CICD</figcaption>
</figure>

## Amplify CLI and Amplify Console

1. **Amplify CLI**: The Amplify Command Line Interface (CLI) provides a convenient way to initialize, configure, and manage Amplify projects from the command line. It offers a simple and consistent interface to set up services, configure authentication, and deploy your application.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/5cli.webp>
<figcaption>CLI</figcaption>
</figure>

2. **Amplify Console**: The Amplify Console provides a hosting and continuous deployment service specifically designed for Amplify applications. It automatically deploys updates as you push changes to your repository, simplifying the deployment process and ensuring a smooth development workflow.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/6console.webp>
<figcaption>Console</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-14-Exploring-the-Power-of-AWS-Amplify-Simplifying-Web-and-Mobile-App-Development/7save.webp>
<figcaption>Save</figcaption>
</figure>

## Benefits of AWS Amplify

1. **Rapid Development**: Amplify's pre-built UI components, simplified workflows, and ready-to-use backend services enable developers to quickly prototype and develop applications.

2. **Scalability and Security**: Leveraging AWS services, Amplify ensures that your applications can scale seamlessly as demand increases while maintaining a high level of security.

3. **Platform Flexibility**: Amplify supports multiple platforms, including web, iOS, Android, and React Native, providing developers with the flexibility to build applications for various devices and environments.

4. **Developer Experience**: With its intuitive interface, extensive documentation, and community support, Amplify offers a pleasant development experience, enabling developers to focus on building great applications.

## Conclusion

AWS Amplify empowers developers to build feature-rich and scalable web and mobile applications by simplifying the development process and integrating with AWS services. Its comprehensive set of tools, including authentication, data storage, APIs, and serverless functions, along with the Amplify CLI and Amplify Console, accelerate the development and deployment cycles. Embrace the power of AWS Amplify to streamline your application development process and deliver exceptional user experiences in a shorter time frame.

_Article Photo by [AWS Documentation](https://docs.aws.amazon.com/)_


Exploring the Power of AWS Amplify - Simplifying Web and Mobile App Development

Jetpack Compose UI Architecture


# Introduction

In a world that is rapidly digitalizing, the need for a secure, transparent, and decentralized mode of operations is ever-increasing. Enter the realm of Blockchain, a groundbreaking technology that has revolutionized industries across the globe. But how does it connect to mobile applications? Let's delve deeper.

Blockchain, at its core, is a distributed ledger that maintains a continuously growing list of records, called blocks, which are secured using cryptography. The most compelling feature of Blockchain is its invulnerability to data modification, instilling the technology with trust, transparency, and security.

Riding on the backbone of Blockchain, we come across the popular Web3. Web3 or Web 3.0, represents the new era of the internet - one that is built on the principles of decentralization. Web3 aims to create an online environment where users maintain control over their own data and interactions, a concept that is inherently symbiotic with Blockchain.

Based on that, we can tell that the integration of Blockchain and Web3 into mobile apps holds tremendous potential and is set to revolutionize various industries. Some key areas where we can expect to see the impact could be: decentralized finance, in-app digital identity or decentralized social networks, among others.

Therefore, we realized the potential of using this technology in mobile apps and decided to catch up within our team. Basically, we decided to create an initial, simple project where we could establish an environment for experimentation and gain practical experience with Blockchain. Furthermore, we aimed to leverage this experience for future projects.

In the following sections, we will introduce you to some crucial terminologies associated with Blockchain and share the idea behind our city hall app leveraging Blockchain. In the future, we will publish a second post to discuss its development process and technological background.

This journey will provide you insights into the future of Blockchain technology within the realm of mobile apps, and how it's set to revolutionize the way we engage with digital platforms. So, get ready for an intriguing exploration into the world of Blockchain and mobile applications. Let's delve in!

# Some new terminologies

### dApps

dApps stands for **Decentralized Applications**. dApps are software applications that run on a decentralized network, typically utilizing blockchain technology. Unlike traditional applications that rely on centralized servers, dApps operate on a network where data and transactions are distributed across multiple nodes. A node is a computer or device that maintains a copy of the entire blockchain and verifies the validity of transactions.

Key features:

- **Decentralization**: dApps operate on a decentralized network, which means there is no single point of control or failure. The application's logic and data are stored and processed on a blockchain or similar distributed ledger technology.

- **Open Source**: dApps are often open source, meaning their source code is publicly available and transparent.

- **Tokenization**: Many dApps have their own native tokens or cryptocurrencies that facilitate transactions within the application or represent ownership rights and value.

- **Smart Contracts**: A smart contract in dapps is an automated, self-executing program that enforces predefined rules on a blockchain.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image1.png>
<figcaption>Image of dApps as decentralized applications, source: https://cryptoadventure.com/wp-content/uploads/2020/06/dapps-developpment.jpg</figcaption>
</figure>

In this article, we describe the development of a simple dApp with **decentralized** features and **smart contracts**.

### Smart Contract

A smart contract is a computer program that automates, verifies, and enforces the terms of a contract or agreement between multiple parties. It is a self-executing contract with the terms and conditions directly written into code.

The key characteristics of a smart contract include:

- **Automation**: A smart contract automatically executes actions or transactions when predefined conditions are met. It eliminates the need for intermediaries or centralized authorities, as the code itself enforces the terms.

- **Digital and Immutable**: Smart contracts exist in a digital form, written in code, and are stored on a blockchain. Once deployed, the code becomes immutable, meaning it cannot be altered or tampered with.

- **Decentralization**: Smart contracts operate on a decentralized network, such as a blockchain, where multiple participants validate and record transactions. This decentralized nature ensures transparency and removes the need for a single central authority.

- **Trust and Security**: Smart contracts leverage cryptographic techniques to secure transactions and ensure that parties involved can trust the execution and outcome of the contract. The decentralized consensus mechanism of the blockchain adds an extra layer of security.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image2.webp>
<figcaption>Example of a Smart contract deployed in Polygonscan</figcaption>
</figure>

For the project described in this article, in order to deploy smart contracts and add records to the blockchain we created a **wallet** in [Metamask](https://metamask.io/).

### Wallet

A Web3 wallet is a cryptographic tool that empowers individuals to exercise control over their digital identity, assets, and interactions in the decentralized landscape. It encapsulates a combination of secure key management, authentication mechanisms, and user-friendly interfaces, allowing users to securely store, send, receive, and manage their blockchain-based assets while seamlessly interacting with dApps and blockchain networks.

For the project described in this article we created a Wallet in Metamask. Metamask functions as a bridge between the user and the blockchain network by a browser extension.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image3.webp>
<figcaption>Metamask, https://metamask.io/</figcaption>
</figure>

All operations that triggered a modification in the blockchain, including the creation of the smart contract, transfers of tokens, or addition of records, are recorded in **transactions** that are visible in the Metamask Wallet.

### Transactions

A transaction is an operation executed in a blockchain. Such operations involve interacting with a smart contract, the contract's code is executed according to the specified function call and parameters. The smart contract may modify its internal state, update balances, emit events, or trigger further actions based on the transaction's logic.

When a transaction is triggered on a blockchain network, several steps occur:

1. **Initiation and Validation**: The transaction is created and initiated by a user or an application. Then, the transaction is broadcasted to the blockchain network, where it undergoes a validation process. Miners (in a proof-of-work network) or validators (in a proof-of-stake network) verify the transaction's validity by checking if the sender has sufficient funds, the transaction format is correct, and any additional conditions imposed by the network's consensus rules are met.

2. **Inclusion in a Block**: Once the transaction is validated, it is grouped together with other validated transactions to form a block. The block contains a collection of transactions and serves as a permanent record on the blockchain.

3. **Mining or Block Finalization**: Mining in a blockchain is the process of validating transactions and adding them to the blockchain while earning rewards for doing so. On the other hand, miners in a blockchain are participants who use computational power to verify and process transactions, adding them to the blockchain and ensuring the network's security and consensus. They are rewarded with cryptocurrency for their efforts. In a proof-of-work network, miners compete to solve a cryptographic puzzle, which requires computational effort. The first miner to solve the puzzle adds the new block (containing the transaction) to the blockchain. In a proof-of-stake network, validators take turns adding blocks based on their stake and selection algorithms.

4. **Consensus and Confirmation**: The newly added block is propagated to other nodes in the network, and they validate and confirm its inclusion. Confirmations represent the number of subsequent blocks added to the blockchain on top of the block containing the transaction. The more confirmations a transaction receives, the higher its finality and security.

5. **Updating the Blockchain State**: After the transaction is executed, the state of the blockchain is updated to reflect the changes. This includes updating account balances, modifying the state of the smart contract, and recording any relevant events or logs associated with the transaction.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image4.webp>
<figcaption>List of transactions of a smart contract</figcaption>
</figure>

In the project described in this article, only two kinds of transactions were executed:

- When deploying a smart contract.

- When adding Pension and Tax records to the blockchain by invoking a method that updated the state in the blockchain.

### Our Idea: A City Hall residence tax records app

The project in this blog post is a mobile application that allows users to verify their pension and tax records stored on the blockchain. For easy understanding, we called the app Manekin which stands for **Money + Kin** (Kin means money in Japanese, then マネー + 金 =マネキン) .

We thought that this kind of app would be a potential idea to be developed on a blockchain-based system instead of a typical cloud-based system in the upcoming years. The reason is that by storing pension and tax records on a blockchain, trust and security will be enhanced, because the data retrieved from the blockchain is immutable and reliable. However, it is important to note that blockchain technology may not be suitable for all types of applications, and in a real product, many constraints such as scalability and performance should be taken into account when designing a dApp.

These are the main features of our app:

- **User authentication**:
  - The app provides a login process where users can authenticate by entering their user id.
  - The user id can be entered by scanning a QR code or manually through a text field.
  - User authentication ensures that only authorized individuals can access their specific records.
  - Logout process, stops access to the records until the next login.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image5.webp>
<figcaption>Mock up of our app idea</figcaption>
</figure>

- **Tax and pension records retrieval**:
  - The app invokes a method in the smart contract, which interacts with the blockchain and retrieves the records associated with the user id.
  - The retrieved records include details such as the paid amount, timestamp, contribution type (tax or pension) and a note with relevant information about the record.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-07-12-Empowering-Mobile-Applications-through-Blockchain-Our-Journey-Part-1/image6.webp>
<figcaption>Mock up of our app idea</figcaption>
</figure>

- **User-friendly interface**:
  - We offer an intuitive and user-friendly interface with easy-to-navigate screens.
  - The application communicates with a smart contract and is deployed on a Blockchain Test network. This all goes under the hood and the user has the experience as a normal database.
  - By providing a user-friendly interface and seamless integration with the smart contract, the app enables users to securely access and review their records.

# Beyond a City Hall app

The above idea is an idea from the digital identity and security field, where users have control over their personal data and information is more secure without relying on traditional centralized authentication systems. Digital identity and security is one of the areas expected to see an impact using Blockchain technology in the near future. Other examples from other areas could be:

1. **Decentralized Finance (DeFi)**: A bank app enabling users to access decentralized financial services, such as lending, borrowing, and trading cryptocurrencies, directly from their smartphones. Blockchain-based smart contracts ensure transparency and security in financial transactions, allowing for peer-to-peer interactions without intermediaries.

2. **Supply Chain Management**: A fashion brand integrated with blockchain technology. This one could track and verify the provenance and authenticity of products throughout the supply chain. This transparency helps in combating counterfeit goods, ensuring fair trade practices, and enhancing consumer trust.

3. **Decentralized Social Networking**: Web3-powered mobile apps can enable decentralized social networking, where users have control over their data and interactions. They can participate in social platforms without concerns about data breaches or censorship, as content is distributed across a decentralized network.

4. **Voting and Governance**: An elections app could enable secure and transparent voting systems, ensuring the integrity of elections and decision-making processes. Web3 principles of decentralization and immutability enhance trust and participation in governance.

The integration of Blockchain and Web3 into mobile apps expands the possibilities for user-centric, secure, and transparent experiences. As the technology evolves and user adoption increases, we can expect to witness a paradigm shift in how we interact with mobile applications, empowering individuals and transforming various industries.

# To be continued...

In conclusion, the integration of Blockchain and Web3 into mobile apps marks an exciting frontier in technology and innovation. The decentralized nature of blockchain technology, coupled with the principles of Web3, empowers individuals with greater control over their data, privacy, and digital interactions.

The potential of Blockchain and Web3 in mobile apps is vast, and as the technology advances, we can anticipate even more innovative use cases. Developers and entrepreneurs are increasingly exploring these possibilities, paving the way for a decentralized future where individuals have greater control and ownership over their digital lives.

In the next blog post, we will delve deeper into the development phase and how we did it to develop an app using this new evolving technology. Stay tuned to learn more about it.


Empowering Mobile Applications through Blockchain: Our Journey, Part 1


Welcome to Flutter calculator app tutorial blog for desktop! In this tutorial, I will guide you through the process of creating a fully functional calculator application using Flutter, Google's powerful framework for building cross-platform applications.

Calculators are essential tools that are used by millions of people every day, and by creating a calculator app using Flutter, you'll not only learn the fundamentals of desktop app development but also gain valuable insights into building user-friendly and efficient interfaces.

Throughout this tutorial blog, we will start from scratch and cover all the necessary steps to develop a feature-rich calculator app. We'll assume you have a basic understanding of Flutter, but even if you're new to Flutter, don't worry! I will explain the concepts and code snippets in a beginner-friendly manner.

Our journey will begin by setting up the development environment for Flutter desktop app development. I will guide you through the installation process, ensuring that you have all the necessary tools and dependencies ready to start coding your calculator app.

Next, we'll dive into the design and layout of the calculator user interface. We'll explore various Flutter widgets and layout options to create a visually appealing and intuitive UI for our calculator. You'll learn how to handle user input, process mathematical operations, and display results in real-time.

Throughout the tutorial, we'll focus on best practices for code organization, reusability, and maintainability. You'll learn how to structure your codebase effectively, separate concerns, and utilize Flutter's widget composition model to build a scalable and extensible calculator app.

By the end of this tutorial series, you'll have a solid understanding of how to build a complete calculator application for desktop using Flutter. You'll be equipped with the skills to extend and customize your calculator app further or even venture into building other types of desktop applications.

So, get ready to embark on this exciting journey of Flutter desktop app development, where you'll learn, create, and have fun building your very own calculator app. Let's dive into the world of Flutter!

## Setting Up the Development Environment

Enabling Configuration:

Go to your root folder in your project and type in the following command for macOS:
```
flutter config --enable-macos-desktop
```
For Linux:
```
flutter config --enable-linux-desktop
```

For Windows:
```
flutter config --enable-windows-desktop
```
The terminal will ask you to restart; after restarting there would be no changes.
Now type the following command in the terminal:

`flutter create .`

and then run the command:

`flutter run -d macos`

You can replace macos according to your platform
Once the command is run, the macos, linux and windows folders are seen and a screen is displayed as below:

![]({{ sitePrefix }}/assets/img/articles/2023-07-05-Desktop-App-in-Flutter/initial_screen.webp)

## Designing the User Interface

### The state class

The `CalculatorState` class represents the state of a calculator in a Flutter application. It contains two properties: `input` and `result`.

`input`: Represents the current input entered by the user in the calculator. It is a String type that holds the user's input, such as numbers, operators, or mathematical expressions.

`result`: Represents the calculated result based on the user's input. It is also a String type that holds the result of the calculations performed on the input.

The `CalculatorState` class has a constructor that initializes the input and result properties. The constructor takes optional named parameters `input` and `result`, which default to empty strings ('') if not provided.

```dart
class CalculatorState {
  final String input;
  final String result;

  CalculatorState({this.input = '', this.result = ''});
}
```

### Button widget 
The `Button` class is a custom Flutter widget that represents a calculator button with customizable properties. Let's go through the different parts of the code to understand its functionality:

- Properties:
  - `buttonColor`: Represents the background color of the button.
  - `textColor`: Represents the color of the button text.
  - `buttonText`: Represents the text displayed on the button.
  - `buttontapped`: Represents a callback function that will be invoked when the button is tapped.

- Constructor:
  - The `Button` class has a constructor that takes named parameters:
    - `buttonColor`, `textColor`, `buttonText`: Optional parameters that allow you to customize the button's appearance.
    - `buttontapped`: A required parameter that takes a `VoidCallback`, which is a function that doesn't return a value and is called when the button is tapped.
    
With this `Button` widget, you can create buttons in your Flutter UI by providing the desired properties, such as `buttonColor`, `textColor`, `buttonText`, and `buttontapped` callback. This allows you to create reusable buttons with customizable styles and behavior.

```dart
import 'package:flutter/material.dart';
class Button extends StatelessWidget {

  final Color? buttonColor;
  final Color? textColor;
  final String? buttonText;
  final VoidCallback buttontapped;

  const Button({
    Key? key,
    this.buttonColor,
    this.textColor,
    required this.buttonText,
    required this.buttontapped,
  }) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return GestureDetector(
      onTap: buttontapped,
      child: Padding(
        padding: const EdgeInsets.all(6),
        child: ClipRRect(
          child: Container(
            decoration: BoxDecoration(
              borderRadius: BorderRadius.circular(56),
              color: buttonColor ?? Colors.grey,
            ),
            child: Center(
              child: Text(
                buttonText ?? '',
                style: TextStyle(
                  color: textColor ?? Colors.black,
                  fontSize: 25,
                  fontWeight: FontWeight.bold,
                ),
              ),
            ),
          ),
        ),
      ),
    );
  }
}

```

### Calculator screen

The list of buttons which will be included in the calculator :

```dart
final List<String> buttons = [
'C',
'+/-',
'%',
'DEL',
'7',
'8',
'9',
'/',
'4',
'5',
'6',
'x',
'1',
'2',
'3',
'-',
'0',
'.',
'=',
'+',
];


```


The following code snippet represents the building of a `GridView` with buttons inside a `Container`. Let's break it down:

```dart
Container(
  child: GridView.builder(
    itemCount: buttons.length,
    gridDelegate: const SliverGridDelegateWithFixedCrossAxisCount(
      crossAxisCount: 4,
    ),
    itemBuilder: (BuildContext context, int index) {
      if (index == 0) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 1) {
        return Button(
          buttonText: buttons[index],
          buttontapped: () {},
        );
      } else if (index == 2) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 3) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 18) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
          buttonColor: Colors.orange,
          textColor: Colors.white,
        );
      } else {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
          buttonColor: isOperator(buttons[index])
              ? Colors.orange
              : Colors.white30.withOpacity(0.3),
          textColor: isOperator(buttons[index])
              ? Colors.white
              : Colors.black,
        );
      }
    },
  ),
),
```
The below grid view is used to create the Calculator screen:

```dart
Container(
  child: GridView.builder(
    itemCount: buttons.length,
    gridDelegate: const SliverGridDelegateWithFixedCrossAxisCount(
      crossAxisCount: 4,
    ),
    itemBuilder: (BuildContext context, int index) {
      if (index == 0) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 1) {
        return Button(
          buttonText: buttons[index],
          buttontapped: () {},
        );
      } else if (index == 2) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 3) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
        );
      } else if (index == 18) {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
          buttonColor: Colors.orange,
          textColor: Colors.white,
        );
      } else {
        return Button(
          buttontapped: () {},
          buttonText: buttons[index],
          buttonColor: isOperator(buttons[index])
              ? Colors.orange
              : Colors.white30.withOpacity(0.3),
          textColor: isOperator(buttons[index])
              ? Colors.white
              : Colors.black,
        );
      }
    },
  ),
),
```

This code creates a `GridView` with a dynamically generated list of buttons based on the `buttons` list. Here's how it works:

- The `GridView.builder` widget is used to build the grid dynamically based on the number of buttons in the `buttons` list.

- The `itemCount` property of `GridView.builder` is set to the length of the `buttons` list.

- The `gridDelegate` property is set to `SliverGridDelegateWithFixedCrossAxisCount` with `crossAxisCount` set to 4, which means there will be 4 buttons per row.

- The `itemBuilder` callback is responsible for building each item in the grid. It takes the `BuildContext` and `index` as parameters.

- Inside the `itemBuilder` callback, there is a series of conditions to determine the type of button to display based on the `index`.

- When `index` is 0, 1, 2, or 3, it represents specific buttons like 'C', '+/-', '%', and 'DEL'. For these buttons, a `Button` widget is created with the corresponding label and an empty `buttontapped` callback.

- When `index` is 18, it represents the '=' button. This button is created with a specific button color (orange) and text color (white). Again, an empty `buttontapped` callback is provided.

- For all other indices, regular buttons are created. The `buttonColor` and `textColor` are set based on whether the button is an operator or not. The `isOperator` function is called to determine this.

- The `buttontapped` callback for all the regular buttons is empty, as specified by `() {}`. You can replace this with the desired logic to handle button taps.

Overall, this code snippet generates a grid of buttons inside a `Container` based on the `buttons` list, with specific customization for certain buttons and dynamic styling based on whether a button is an operator or not.

```dart
 bool isOperator(String x) {
    if (x == '/' || x == 'x' || x == '-' || x == '+' || x == '=') {
      return true;
    }
    return false;
  }
```

The `isOperator`  method is a helper function used to determine if a given button label is an operator (/, x, -, +, =).
It returns true if the button label is an operator, and false otherwise.

The code below is the input space and the result space:

```dart
Expanded(
            child: SizedBox(
              child: Column(
                mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                children: <Widget>[
                  Container(
                    padding: const EdgeInsets.all(20),
                    alignment: Alignment.centerRight,
                    child: Text(
                      '',
                      style: const TextStyle(fontSize: 18, color: Colors.white),
                    ),
                  ),
                  Container(
                    padding: const EdgeInsets.all(15),
                    alignment: Alignment.centerRight,
                    child: Text(
                      '',
                      style: const TextStyle(
                          fontSize: 30,
                          color: Colors.white,
                          fontWeight: FontWeight.bold),
                    ),
                  )
                ],
              ),
            ),
          ),
```

## View model

The provided code defines a CalculatorViewModel class, which extends StateNotifier<CalculatorState>. This class is responsible for managing the state and logic of a calculator application. Let's break down the code and understand its functionality:

```dart
class CalculatorViewModel extends StateNotifier<CalculatorState> {
  CalculatorViewModel() : super(CalculatorState());
```

The clear method clears the calculator's input and sets the result to '0'. It updates the state by assigning a new instance of CalculatorState using state = CalculatorState(input: '', result: '0').
```dart
  void clear() {
    state = CalculatorState(input: '', result: '0');
  }
```
The equalPressed method is responsible for evaluating the input expression and updating the state with the calculated result.
- It retrieves the input from the state and replaces 'x' with '*' to match the expression format.
- The expression is then parsed using the Parser class from math_expressions.
- The parsed expression is evaluated using the Expression class and a ContextModel.
- The result is obtained as a double value, converted to a string, and stored in the answer variable.
- Finally, the state is updated with the modified input expression and the calculated result.

```dart
  void equalPressed() {
    String finaluserinput = state.input;
    finaluserinput = state.input.replaceAll('x', '*');

    Parser p = Parser();
    Expression exp = p.parse(finaluserinput);
    ContextModel cm = ContextModel();
    double eval = exp.evaluate(EvaluationType.REAL, cm);
    final answer = eval.toString();
    state = CalculatorState(input: finaluserinput, result: answer);
  }
```

The updateInput method is responsible for appending the given operation (button label) to the calculator's input.
It retrieves the current input from the state, concatenates the operation, and assigns the updated input to the input variable.
The state is then updated with the modified input while keeping the result unchanged.

```dart
void updateInput(String operation) {
var input = state.input;
input += operation;
state = CalculatorState(input: input, result: state.result);
}
```

The delete method removes the last character from the calculator's input.
It retrieves the current input from the state and creates a substring of the input excluding the last character.
The modified input is assigned to the finalInput variable.
The state is then updated with the modified input while keeping the result unchanged.

```dart
  void delete() {
    var input = state.input;
    final finalInput = input.substring(0, input.length - 1);

    state = CalculatorState(input: finalInput, result: state.result);
  }
```

The CalculatorViewModel class encapsulates the logic for handling button presses, performing calculations, and managing the state of the calculator. It utilizes the state property inherited from StateNotifier to update the state and notify listeners of state changes.

## Connecting the UI with the view model

The provided code defines a `calculatorProvider` using the `StateNotifierProvider` from the `flutter_riverpod` package. This provider is responsible for creating and managing an instance of the `CalculatorViewModel` class and providing access to its associated state of type `CalculatorState` to other parts of the application.

Here's an explanation of the code:

```dart
final calculatorProvider =
    StateNotifierProvider<CalculatorViewModel, CalculatorState>(
  (ref) => CalculatorViewModel(),
);
```

- The `calculatorProvider` is declared as a constant using the `final` keyword.
- It is assigned the result of the `StateNotifierProvider` constructor, which takes two type parameters: `CalculatorViewModel` and `CalculatorState`.
- The first type parameter, `CalculatorViewModel`, specifies the type of the state notifier being provided. In this case, it is an instance of the `CalculatorViewModel` class.
- The second type parameter, `CalculatorState`, specifies the type of the state managed by the state notifier. Here, it is the `CalculatorState` class.
- The constructor argument `(ref) => CalculatorViewModel()` is a callback function that gets executed when the provider is first accessed. It creates a new instance of the `CalculatorViewModel` class and returns it. The `ref` parameter provides access to the provider's container and can be used to read other providers or perform additional setup.

With the `calculatorProvider` defined, other parts of the application can access the `CalculatorViewModel` instance and its associated state using the `ProviderContainer` or by using the `ConsumerWidget` or `Consumer` widget provided by `flutter_riverpod`. This allows components to interact with the calculator's state and trigger updates based on user interactions or changes in the state.


To acees the state and reflect it in the UI we will add the below in the build function
```dart
final viewModel = ref.read(calculatorProvider.notifier);
final state = ref.watch(calculatorProvider);
```

After bulilding and running the project, the final app would look like this:
![]({{ sitePrefix }}/assets/img/articles/2023-07-05-Desktop-App-in-Flutter/final_img.webp)


The full code can be found here:
[GithubRepo](https://github.com/huma11farheen/desktop_app)

_Article Photo by [Moataz Nabil](https://bitrise.io/blog/post/build-and-deploy-a-flutter-desktop-app-for-linux)_

Building desktop app with flutter : Tutorial


Launched on May 24, 2023, Microsoft Fabric is an innovative cloud-based SaaS analytics platform that streamlines data analytics by integrating multiple workloads into one comprehensive solution. Fabric covers a wide range of areas, including data ingestion, data warehousing, and data engineering, and represents a bold step toward the future of data analytics.

Previously, Microsoft's data analytics tools—Data Factory, Synapse, Power BI, Purview, and others—operated as individual services. That meant users had to set up, pay for, and connect to each tool individually. Such an approach complicated the end-to-end data process and increased costs. Microsoft Fabric, however, offers a strategic solution by combining these services into one integrated system, providing a more user-friendly and cost-effective solution. It provides a comprehensive range of fully integrated analytical experiences, each tailored to a unique task and user profile and addressing an end-to-end analytical requirement. Data Engineering, Azure Data Factory, Data Science, Data Warehouse, Real-Time Analytics, and Power BI are all included.

## An overview of the core features

Microsoft Fabric can be seen as an all-in-one platform that gathers all of the necessary components for data analysis and is developed with various user roles in mind. It consists of several components, all of which are designed to make dealing with data easier. Microsoft has added Azure Data Factory in Fabric as a connector, bringing information from over 200 different places together in a simple and powerful way.

While Fabric's Data Warehouse functions as a sizable storage facility that houses all of your data efficiently and conveniently, Fabric's Data Science component, which is completely integrated with Azure Machine Learning, functions like a science lab for tech specialists to develop and deploy their data prediction models. Also available are real-time analytics and Power BI, a user-friendly tool for data visualization that enables anybody to access, comprehend, and base decisions on data. Last but not least we have Microsoft Fabric Spark integration with Data Factory which enables the scheduling and orchestration of Spark notebooks and jobs.

Let's take a closer look at some of Microsoft Fabric's standout features and have a quick debate on whether you should switch over immediately to this cutting-edge new data and analytics platform from Microsoft.

![Microsoft Fabric Overview]({{ sitePrefix }}/assets/img/articles/2023-06-15-Embracing-Future-Data-Analytics-Microsoft-Fabric/fabric_ml.webp)

## Storing and managing data with OneLake

OneLake, a layer that lies above Azure Data Lake Storage Gen-2, is a fundamental component of Fabric. It transforms data lake management by consolidating several storage accounts from many geographical regions into a single user-friendly layer. Furthermore, whenever a new workspace is created, OneLake immediately establishes a new storage area within the existing Data Lake, ensuring easy access, speed, and compliance with data governance rules.

OneLake is a Fabric component that simplifies how organizations store and retrieve data. Consider it a gigantic virtual filing cabinet that consolidates data from various locations into a single, easily accessible location. This makes finding and using data easier while also keeping it secure.

## Improved collaboration with Workspaces

Fabric's use of workspaces, which might be familiar to you from Power BI, operating as a centralized access point within Fabric, is one of its notable features. These workspaces provide secure, specialized areas for certain groups to store, distribute, and manage data and project components, which are referred to as "artifacts." This novel solution allows several users to collaborate inside the same workspace while using the same data, without the requirement for data duplication or relocation between environments or cloud services.

Fabric workspaces are similar to digital rooms where teams may store and share data. They're similar to the shared folders that many of us use every day, but they're built to manage large amounts of data and give teams more control over who can access them.

## Keeping your data secure with OneSecurity

OneSecurity, a Fabric security feature, delivers full workspace-level security that integrates seamlessly with Power BI. Governance and compliance measures are already built into Azure Data Lake Storage and Power BI. Furthermore, Microsoft's Purview is intended to support Fabric's governance and compliance.

Simply defined, OneSecurity is a Fabric feature that aids in data security. It operates in the background to guarantee that only authorized users have access to the data they require. This capability, combined with Microsoft's robust data governance and compliance policies and tools, helps keep your data safe and secure.

## An overview of pricing and licenses

In terms of pricing and licensing, Microsoft is following the Power BI Premium strategy with a capacity licensing model for Fabric. Customers buy Fabric Quality Units (FQUs), which give a common degree of performance capacity for all tools. Although this strategy is adaptable, details concerning scaling remain unknown. Furthermore, the licensing restrictions for Power BI continue to apply within Fabric.

In a way, Fabric employs a 'pay for what you use' model. Customers purchase 'units' of capacity, which govern how much work they may perform on the platform. However, the pricing details are a little complicated, and Microsoft hasn't provided all of them yet. However, businesses may find it more cost-effective than the alternatives, if they utilize it wisely and take advantage of its capacity to simplify.

## Should I move my data architecture to Microsoft Fabric?

With the release of Fabric, existing customers of Microsoft's data analytics products may question the need for migration. While immediate migration is not required, Microsoft's emphasis on Fabric indicates that it may become the preferred approach in the future. Current Power BI customers will likely find the transfer easiest, as enabling Fabric provides instant access to powerful capabilities such as Synapse Analytics, Data Science, and Data Engineering services.

So, if you're already utilizing Microsoft's data tools, you might be wondering if you need to migrate to Fabric. The quick answer is no, not right away. Microsoft has stated that it will continue to support earlier technologies, but it is focusing more on Fabric, implying that it may become the primary way to handle data in the future.

Another thing to consider, just like any other SaaS offering, is that it’s cloud-based. If you are still using on-premises architectures due to legal, compliance, or just internal decisions, keep in mind the cost and effort of moving to the cloud.

At Monstarlab, we have a specialized data and analytics team ready to support you on all types of investments you might be considering, being just a simple discovery use case, a cloud migration one, or a lift and shift from your existing Azure-based data architecture to a new Microsoft Fabric based one.

Thank you,
Rui Machado


Embracing the Future of Data Analytics with Microsoft Fabric


In today's fast-paced world, creating robust and user-friendly mobile applications is essential for businesses and developers alike. However, the process of building an app can be complex and time-consuming. That's where FlutterFlow comes in. FlutterFlow is a powerful visual development platform that simplifies the app development process using Google's Flutter framework. In this article, we'll explore the features and benefits of FlutterFlow and how it can revolutionize your app development workflow.

### What is FlutterFlow?
FlutterFlow is an intuitive and powerful visual development platform that allows developers to design, prototype, and build mobile applications using a visual interface. It leverages the power of Google's Flutter framework, a cross-platform UI toolkit, to create high-quality native apps for iOS and Android.

### Amazing Visual Interface
One of the standout features of FlutterFlow is its visual interface, which enables developers to design app screens and create interactive prototypes without writing a single line of code. With its drag-and-drop functionality, you can quickly build UI layouts, add components, and define interactions between screens. This visual approach makes it easy to iterate and refine your app's design, saving time and effort in the development process.

### Easy Code Generation
While FlutterFlow simplifies the visual design aspect of app development, it doesn't limit you to visual-only workflows. FlutterFlow generates clean, production-ready Flutter code based on your designs, giving you the flexibility to customize and extend your app's functionality using traditional coding practices. This allows developers to seamlessly transition from the visual interface to writing code, providing the best of both worlds.

### Element Library
FlutterFlow comes with an extensive library of pre-built UI components that you can use to create stunning app interfaces. From buttons and input fields to complex elements like lists and grids, FlutterFlow offers a wide range of ready-to-use components. This saves developers from reinventing the wheel and accelerates the development process.

### Real-Time Collaboration
Collaboration is key when working on app development projects, and FlutterFlow offers real-time collaboration features that streamline teamwork. Multiple team members can work simultaneously on the same project, making it easy to share ideas, gather feedback, and track progress. With FlutterFlow's collaboration capabilities, you can ensure efficient communication and maximize productivity.

### Integration with Backend Services
Building a complete app often involves backend integration, and FlutterFlow offers seamless integration with popular backend services. This allows developers to easily connect their apps to databases, authenticate users, and fetch data from APIs. These built-in integrations simplify the process of incorporating backend functionality, such as working with services like Firebase, GraphQL, and REST APIs, and enable robust functionality in your app. However, it's worth noting that there are scenarios where an app can be considered complete and fully functional without the need for complex backend code or integration.

## To use FlutterFlow, follow these steps

1. Sign up and Create a Project:
    * Go to the [FlutterFlow website](https://flutterflow.io/) and sign up for an account.
    * Once you're signed in, create a new project by clicking on the "Create Project" button.

2. Design Your App:
    * After creating a project, you'll be directed to the FlutterFlow visual editor.
    * Start designing your app's screens by dragging and dropping UI components from the sidebar onto the canvas.
    * Customize the appearance of the components, define layouts, and add interactions between screens using the visual interface.
    
3. Configure Data Sources:
    * If your app requires data from a backend service or API, you can configure data sources in FlutterFlow.
    * Connect your app to external services like Firebase, REST APIs, or GraphQL by setting up the necessary configurations within FlutterFlow.

4. Add Logic and Functionality:
    * While FlutterFlow simplifies visual design, you can also add logic and functionality to your app using code blocks.
    * Utilize FlutterFlow's built-in functionality blocks or create custom logic using Dart code snippets.

5. Preview and Test:
    * Use the Preview mode in FlutterFlow to test your app's design and interactions within the visual editor.
    * Preview your app on different device sizes and orientations to ensure responsiveness.

6. Generate Flutter Code:
    * Once you're satisfied with your app's design and functionality, you can generate the Flutter code.
    * Click on the "Export Code" button to generate clean and structured Flutter code based on your FlutterFlow project.

7. Import to Flutter:
    * Open your preferred Flutter development environment (such as Android Studio or Visual Studio Code) and create a new Flutter project or open an existing one.
    * Copy the generated Flutter code from FlutterFlow and replace the default code in your Flutter project with the generated code.

8. Build and Launch Your App:
    * Build and run your Flutter app using the Flutter CLI commands (`flutter run`) or through your integrated development environment.
    * Test your app on different devices or emulators to ensure it functions as intended.

## Let's build a simple screen using FlutterFlow

Go to the FlutterFlow website (https://flutterflow.io/) and sign up for an account.
Image

A prompt to enter the project details along with firebase integration will be displayed, for this blog post I will not integrate Firebase.

![]({{ sitePrefix }}/assets/img/articles/2023-05-29-Streamlining-App-Development-with-FlutterFlow/create-project.webp)


### The tools
FlutterFlow provides a set of tools and panels on the left and right sides of the interface to assist with the visual development process. Let's take a closer look at each side and their respective tools:

The tools on the left panel:

* Screen Selection: At the top left corner, you'll find a dropdown menu that allows you to select and switch between different screens in your app. This is where you can manage and access various screens within your project.

* Component Tree: Below the screen selection dropdown, you'll find the Component Tree panel. It provides a hierarchical view of all the UI components present on the currently selected screen. You can expand and collapse the tree to navigate and select specific components for editing.

* Properties Panel: The Properties panel, located on the left side, displays the properties and attributes of the selected UI component. You can modify these properties to customize the appearance, behavior, and styling of the selected component.

* Data Sources: The Data Sources panel allows you to manage and connect data sources to your app. You can define data models, integrate APIs, and create dynamic content for your app using this panel. It provides a visual interface for managing data and API connections.

* Actions: The Actions panel enables you to define and manage interactive behaviors within your app. You can create events and actions that respond to user interactions, such as button clicks or form submissions. This panel allows you to add logic and interactivity to your app without writing code.

The tools on the right panel:

* Widget Library: The Widget Library panel, located on the right side, provides a comprehensive collection of pre-built UI components that you can drag and drop onto your app screens. It includes various categories of widgets, such as buttons, text fields, images, lists, and more. You can customize these widgets to fit your design requirements.

* Styles: The Styles panel allows you to manage and apply styles to your UI components. It provides options for defining and customizing text styles, colors, spacing, and other visual properties. You can create reusable styles to maintain consistency across your app.

* Asset Manager: The Asset Manager panel lets you upload and manage the assets (images, icons, fonts, etc.) used in your app. You can upload assets, organize them into folders, and easily access and use them within your app screens.

* Component Structure: The Component Structure panel shows a hierarchical overview of your app's components and screens. It provides a visual representation of the entire structure of your app and allows you to navigate between screens and components easily.

* Preview and Simulator: At the top right corner, you'll find buttons to preview your app's design and functionality. You can view your app's appearance on different device sizes and orientations, and even interact with the previewed app using the built-in simulator.

In the below image, I have dragged and dropped a Column as a parent Widget and the pageView component and when you click the component, you will have options to position the elements and design them accordingly.
![]({{ sitePrefix }}/assets/img/articles/2023-05-29-Streamlining-App-Development-with-FlutterFlow/first-step.webp)

Next, I added a Container Widget with Row and three more Container with image Components
![]({{ sitePrefix }}/assets/img/articles/2023-05-29-Streamlining-App-Development-with-FlutterFlow/second-step.webp)

Then, I added a ListView component below and added Cards with Images. The listView is Expanded by selecting the option on the right and making a horizontal scroll.
![]({{ sitePrefix }}/assets/img/articles/2023-05-29-Streamlining-App-Development-with-FlutterFlow/third-step.webp)

Finally, when you are satisfied with the look of the screen select on the run option to preview the app. This is how it would look like
![]({{ sitePrefix }}/assets/img/articles/2023-05-29-Streamlining-App-Development-with-FlutterFlow/final.webp)

## Limitations and areas where FlutterFlow may fall short

While FlutterFlow offers many benefits, it's important to consider its limitations and areas where it may fall short. Here are some downsides and considerations:
* Limited Customization: FlutterFlow provides a visual interface for building UI components, but it may have limitations when it comes to fine-grained customization. Advanced design elements or complex animations may require manual coding or customization outside of the visual builder.

* Learning Curve: While FlutterFlow simplifies the app development process, there is still a learning curve associated with using the tool. It may take time to familiarize yourself with its interface, components, and workflows.

* Less Control and Flexibility: Since FlutterFlow is a low-code tool, it abstracts certain aspects of app development. This can limit the level of control and flexibility you have over your app's code and architecture. Custom or complex requirements may require direct coding in Flutter instead of relying solely on FlutterFlow.

* Production Readiness: While FlutterFlow has improved over time, it may still have some limitations when it comes to production-level app development. It's important to thoroughly test and validate your app's behavior, performance, and scalability when using FlutterFlow.

* Complex Scenarios: While FlutterFlow can handle many common app scenarios, it may struggle with more complicated use cases that require access to device sensors, camera functionality, Bluetooth connectivity, or advanced platform-specific features. In such cases, manual coding or integrating additional Flutter packages may be necessary.

* Accessibility Support: FlutterFlow provides basic accessibility features, but it may not cover all the requirements for building fully accessible apps. Additional manual coding and testing may be needed to ensure proper accessibility support.

* Unit Testing: FlutterFlow's visual builder approach may make it challenging to write comprehensive unit tests. Extensive testing and validation may require writing custom code and utilizing Flutter's testing framework directly.

* Time Savings: The amount of time saved using FlutterFlow versus building from scratch depends on various factors such as the complexity of the app, your familiarity with FlutterFlow, and the level of customization required. While FlutterFlow can accelerate the development process by generating code and providing visual editing capabilities, it may still require additional manual coding for specific requirements, resulting in varying time savings.

## Conclusion:

FlutterFlow is a game-changer for mobile app development. Its visual interface, code generation capabilities, extensive component library, and seamless backend integrations make it a go-to platform for developers looking to build high-quality native apps efficiently. By simplifying the development process and fostering collaboration, FlutterFlow empowers developers to bring their app ideas to life quickly and effectively. Whether you're a beginner or an experienced developer, FlutterFlow is a tool worth exploring to unlock your app development potential.
Mind you, FlutterFlow is a powerful visual development tool that simplifies the app development process, but it's important to have a basic understanding of Flutter and Dart programming concepts to fully leverage its capabilities. FlutterFlow complements your development workflow by providing a visual interface and code generation, but you may need to customize and extend your app's functionality through manual coding.
However, it's important to evaluate its suitability for your specific project's requirements, complexity, and customization needs. Careful consideration should be given to its limitations, production readiness, support for advanced scenarios, accessibility, unit testing, and the trade-offs between time saved and flexibility.

Refer to FlutterFlow's documentation, tutorials, and community resources to explore its features in-depth and make the most of this powerful app development platform.

Remember, the best way to truly experience the power of FlutterFlow is to try it out for yourself. So, dive in.



Streamlining App Development with FlutterFlow: An Introduction


In this article, I will introduce a powerful web server named Traefik, written in the Go language. Compared with Nginx, currently the most popular web proxy server, Traefik stands out for its simplicity and speed.

Here are the official website and source code of Traefik:
- [Traefik official site](https://traefik.io)
- [Traefik GitHub repository](https://github.com/traefik/traefik)

To follow and run the demo provided in this article, you should have:
- Experience using other web servers like Nginx or Apache
- Basic knowledge of Docker and Docker Compose

As described in the Github repository, Traefik has the following features:
- Continuous configuration updates (No restarts!)
- Support for multiple load balancing algorithms
- Provides HTTPS to your microservices by leveraging Let's Encrypt (wildcard certificates support)
- Circuit breakers, retry mechanisms
- A clean, user-friendly web UI
- Readiness for Websocket, HTTP/2, GRPC
- Provides metrics (Rest, Prometheus, Datadog, Statsd, InfluxDB)
- Keeps access logs (JSON, CLF)
- Fast
- Exposes a Rest API
- Packaged as a single binary file and available as an official docker image

I personally recommend Traefik for the following three reasons. I encourage you to consider using it as the proxy server in your next startup project:
- Easier configuration files
- Simpler SSL support
- Faster

Next, I will run a Traefik instance using a Docker container to demonstrate these advantages.

## Simplified Configuration

When compared to Nginx, Traefik's configuration files are simpler to handle. In this section, I will use the following three configuration files to start an instance:
- `docker-compose.yml`
- `traefik.yml`
- `dynamic-traefik.yml`

### Docker

Being written in the Go language, Traefik naturally aligns well with Docker and Kubernetes. In this article, I will use the following Docker Compose file to start a Traefik instance.

```yml
version: "3"

services:
  reverse-proxy:
    image: traefik:v2.5.6
    command:
      - --api.insecure=true
    ports:
      - 80:80
      - 443:443
      - 7000:7000
    volumes:
      - /etc/traefik/:/etc/traefik/
      - /var/log/traefik/:/var/log/traefik/
      - /etc/ssl/certs/:/etc/ssl/certs/
      - /etc/ssl/letsencrypt:/letsencrypt/
    restart: always
```

The following ports are specified in our `yml` file:
- `80`: http
- `443`: https
- `7000`: Visual web panel for easier Traefik management

The following volumes are added in our `yml` file:
- `/etc/traefik:/etc/traefik/`: Stores the configuration files
- `/var/log/traefik/:/var/log/traefik/`: Stores the log files
- `/etc/ssl/certs/:/etc/ssl/certs/`: Stores the SSL certificate files
- `/etc/ssl/letsencrypt:/letsencrypt/`: Stores LetsEncrypt-related files

### Main configuration file

The main configuration file, `traefik.yml`, holds the global configurations of Traefik.
This is similar to the `event`, `server`, and `http` nodes defined in the main configurations of Traefik.
Traefik supports the standard yml format to define the configurations, which is easier to read and edit.

```yml
entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
  metrics:
    address: ":7000"
providers:
  file:
    directory: /etc/traefik/
    watch: true
api:
  dashboard: true
log:
  level: WARN
```

### Dynamic configuration file

The dynamic configuration file, `dynamic-traefik.yml`, contains site configurations.
It is similar to the `include` command in Nginx like the following:

```
include /etc/nginx/vhost/*.conf;
```

In the `dynamic-traefik.yml` file, the domain `test.fczm.site` is bound to a backend server instance `http://10.0.0.1:8080`.
While Traefik can serve as a static web server, it's usually deployed as a proxy server.

```yml
http:
  routers:
    router-test-api:
      entryPoints:
        - "websecure"
      rule: Host(`test.fczm.site`)
      service: test-api

  services:
    test-api:
      loadBalancer:
        servers:
          - url: "http://172.31.1.57:8080"
        healthCheck:
          path: /actuator/health
          interval: "10s"
          timeout: "3s"
```

This demo file is a simple dynamic configuration file, containing two nodes:
- `routers`: Defines the routers, including the domain, SSL certificate, backend service, etc.
- `services`: Defines the specific service, in this case, a backend server.

After preparing the Traefik-related configuration file, simply move it into the folder `/etc/traefik/` defined in the Docker Compose file.

I have a simple Spring Boot backend server set up for this demonstration.
The backend server tested successfully:
```
➜  ~ curl http://172.31.4.26:8070/actuator/health
{"status":"UP"}#
```

Finally, you can launch it with Docker Compose.

## Easier SSL support

With Traefik, it is easy to use the SSL certificates provided by `LetsEncrypt`, which can be automatically updated.
To support `LetsEncrypt` SSL certificates, append a `certificatesResolvers` node should be added to the main configuration file `traefik.yml`.

```yml
certificatesResolvers:
  testResolver:
    acme:
      email: test@fczm.site
      storage: /letsencrypt/acme.json
      httpChallenge:
        entryPoint: web
```

Next, insert the resolver into the routers in the `dynamic-traefik.yml` file, as shown below:

```yml
http:
  routers:
    router-test-api:
      entryPoints:
        - "websecure"
      rule: Host(`test.fczm.site`)
      service: test-api
      tls:
        certResolver: testResolver
```

Upon restarting Traefik and waiting a while, the SSL certificate becomes active.
Manual SSL certificate management is also supported.

## Improved speed

Traefik performs faster than Nginx.
For performance testing, I've prepared the following environment:

| Item | Value |
| --- | --- |
| Server Provider | AWS |
| Region | Tokyo |
| CPU | 1 Core |
| Memory | 2 GB |
| Backend | Spring Boot Demo Application |
| SSL | Required |

I employed the command-line tool `wrk` for the performance tests.

Initially, I set up a Nginx proxy server for the backend instance and executed the following command to test its performance:

```shell
➜  ~ wrk  -t 10 -c 200 -d 60s -T 30s --latency https://test.fczm.site/actuator/health
Running 1m test @ https://test.fczm.site
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   147.38ms  117.23ms   1.80s    80.74%
    Req/Sec   157.30     60.24   380.00     69.43%
  Latency Distribution
     50%  119.11ms
     75%  194.24ms
     90%  290.68ms
     99%  523.39ms
  87393 requests in 1.00m, 30.42MB read
Requests/sec:   1455.54
Transfer/sec:    518.85KB
```

Afterwards, I ran the same script with Traefik:

```shell
➜  ~ wrk -t 10 -c 200 -d 60s -T 30s --latency https://test.fczm.site/actuator/health
Running 1m test @ https://test.fczm.site/actuator/health
  10 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   106.62ms   83.21ms   1.41s    87.12%
    Req/Sec   211.13     58.45   400.00     70.40%
  Latency Distribution
     50%   83.21ms
     75%  133.38ms
     90%  203.80ms
     99%  388.29ms
  123927 requests in 1.00m, 48.81MB read
Requests/sec:   2061.95
Transfer/sec:    831.63KB
```

In this performance test, Nginx sent 87,393 requests within 1 minute, while Traefik sent 123,927 requests during the same period, making it 1.4x faster.

## Conclusion

Traefik is simpler and faster than Nginx.
Written in Go, it naturally integrates with Docker and Kubernetes.
If you're struggling with Nginx, consider trying Traefik in your next project.

_Article Photo by [Traefik](https://traefik.io/blog/announcing-traefik-proxy-2-6/)_


Exploring Traefik: A simpler and faster web server


Smart contracts are an exciting way to build decentralized applications (DApps) on a blockchain. This tutorial will guide you in building your first smart contract on the Ethereum blockchain using Solidity.

## What is a Smart Contract?

According to Wikipedia, a [Smart Contract](https://en.wikipedia.org/wiki/Smart_contract) is a computer program or a transaction protocol stored on the Blockchain network.

You can say it's like an agreement between two people in the form of computer code. Transactions within a smart contract are processed by the blockchain and stored as a character chain hex address with the prefix "0x". This means they can be sent automatically, eliminating the need for a third party.

_Example_: **0xe827a5db9559dce2091c535e3f9ba9409defcfef18936da72c79dac190bfbab9**

**Note**: When your Smart Contract is deployed, everything stored on blockchain can not be change.

## What is Solidity?

Solidity is an object-oriented programming language created specifically by the Ethereum Network team for constructing and designing smart contracts on Blockchain platforms.

- Solidity was proposed in August 2014 by Gavin Wood.
- Solidity is a high-level language for smart contract applications.
- It is an object-oriented language designed specifically for the Ethereum Virtual Machine.
- Solidity is most profoundly influenced by C++, but also borrowed concepts from languages like Python, JavaScript.
- The open source project can be found on [Github](https://github.com/ethereum/solidity)
- All documentation can be accessed [here](https://docs.soliditylang.org/en/v0.8.19/)

## Problem to solve

Firstly, we will **create an amount token**. To do so, please follow the steps below:

- Create the Minter (originator)
- Create the Supply
- Create the Balance (account balance)

Next, we will **create a sent money** function. This involves several key points:

- Identify the Receiver's address (it's a character chain hex address)
- Check if your amount is less than or equal to the balance. If it's not, throw an error message stating, **"Not enough money to make the transaction"**
- If the transaction to send money is successful, we will update the total amount for the sender. This means: the sender's balance will decrease by the amount sent (**balance sender -= amount**). In parallel with this, we will also update the receiver's amount: the receiver's balance will increase by the amount received (**balance receiver += amount**).

## Build your first Smart Contract

1. Open Remix IDE from [here](https://remix.ethereum.org/)
2. Click Accept and get ready
3. Under **default_workspace**, click on _create new file_
4. Set the name to be **FirstSmartContract.sol**

Great, now we're ready to start coding.

**Contract code**

1. In the first line of the Smart Contract, we have to specify the _solidity version_.

```js
    pragma solidity ^0.8.18;
```

2. We then create a contract with the name **FirstSmartContract**.

```cpp
    contract FirstSmartContract {
        ...
    }
```

3. In the **FirstSmartContract {..}** contract, we will input some code, please see below.
4. We will create variables where the smart contract will store the minter [address](https://docs.soliditylang.org/en/v0.8.19/types.html#address) (a 42 character hex string with a "0x" prefix) of the both the Landlord & the Tenant.

```ts
    address public minter;
```

5. Create the **balances** value of type [mapping](https://docs.soliditylang.org/en/v0.8.19/types.html#mapping-types).

```ts
    mapping (address => uint) public balances;
```

This line indicates that for each [address](https://docs.soliditylang.org/en/v0.8.19/types.html#address) input, we will get a value of type **uint**. In this case, the **uint** represents the amount associated with each **address**. Since the balance for each address can never be negative, the value type is **uint** (range from 0 to 4294967295).

6. Create an [event](https://docs.soliditylang.org/en/v0.8.19/structure-of-a-contract.html#events) named _sendSuccess_.

This event requires three input values: the address of the minter, the address of the wallet you want to send to, and finally, the amount you want to send.

```cpp
    event sendSuccess(address from, address to, uint amount);
```

7. In the **constructor**, we will set minter address.

```ts
    constructor() {
        minter = msg.sender;
    }
```

In Solidity, if you want to get the address of the minter (sender) you can use `msg.sender;`.

8. Next, we create the **sendMoney** function. We need to pass two values as input: the **receiver's address** and the **amount sent**.

```ts
    function sendMoney(address receiver, uint amount) public {
        ....
    }
```

Firstly, we need to check whether the amount you want to send is less than your total balance. If this condition is not met, the program should throw the message **Not enough money to make the transaction** and stop.

This check can be implemented with the following line:

```ts
require(amount <
  balances[msg.sender], "Not enough money to make the transaction");
```

If the condition is met, the program will continue to the next lines. The next steps are:

- Update the receiver's total balance:

```ts
balances[receiver] += amount;
```

- Subtract the sent amount from your balance:

```ts
balances[msg.sender] -= amount;
```

**Remember**: For each of your transactions, you will be charged a gas fee, which is regulated by Ethereum.

Almost there.
To register the transaction on the blockchain, we will emit an event:

```ts
    emit sendSuccess(msg.sender, receiver, amount);
```

In summary, the **sendMoney** function will look like this:

```ts
    function sendMoney(address receiver, uint amount) public {
        require (amount < balances[msg.sender], "Not enough money to make the transaction");
        balances[receiver] += amount;
        balances[msg.sender] -= amount;
        emit sendSuccess(msg.sender, receiver, amount);
    }
```

9. Finally, we will create the **mint** function, which is used to generate balance for a given address.

```ts
    function mint(address receiver, uint amount) public {
        balances[receiver] += amount;
    }
```

10. Now, it's time for a demonstration:

## Compile

Click on the `Solidity Compile` option in the left sidebar.

1. Select the compiler version `0.8.18+`
2. Then, click on `Compile FirstSmartContract.sol`

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-compile.webp)

## Deploy

Click on the `Deploy & Run Transactions` option in the left sidebar.

1. Choose `Environment > JavaScript VM (London)`
2. Click on `Deploy`

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-deploy.webp)

🎉 Congratulations, your smart contract has been deployed. 🎉

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-transaction-success.webp)

## Create Sample Transaction

Under `Deployed Contracts` click on `FIRSTSMARTCONTRACT AT... (MEMORY) ` to expand the section and fill in the details.

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-transaction.webp)

We have 4 buttons here:

- The first is **mint**. This button initializes the balance of the input address.
- The second is **sendMoney**. When you input a value in the box next to it and click the `sendMoney` button, a transaction will be executed.
- Next, we have the **balances** variable you already created in the code. Just input the wallet address (e.g., 0x...), and the available balance will be displayed.
- Finally, there's the **minter** variable. The address of the minter will be printed when you click this button.

Remember, your first Ether Address is the landlord of the room. (The one with 99.99 test ether in the wallet.)

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-current-address.webp)

Copy your current account address and input it into the **mint** field, followed by the amount. It should look like this:
Press the **mint** button:

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-mint-current.webp)

To check your balances, paste the account address into the **balances** field and press the `balances` button. You should now see that the balance is **1000**.

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-mint-balances.webp)

Change the **Account Address** from the dropdown menu. (Choose any address except the one with 99.99 ether).

In this example, I will choose the last address.

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-change-account-address.webp)

Copy the address you just selected and paste it into the **sendMoney** field, followed by the amount you want to send. It should look similar to the following:

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-send-money.webp)

After that, change the selected account address to your 1st Ether Address and press the **sendMoney** button.

🎉 Well done. Congratulations, the money has been sent. 🎉

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-send-money-success.webp)

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-25-Build-your-first-smart-contract-with-Ethereum-and-Solidity/solidity-check-money-sent.webp>
<figcaption>Check balances of account address receiver</figcaption>
</figure>

## Gas fee

If you pay close attention, you will notice that you lose a few wei each time you make a transaction. This is called a gas fee.

- 1 ether = 1,000,000,000,000,000,000 wei

The Ethereum gas fee exists to compensate network validators for their work in securing the blockchain and network.

Without these fees, there would be little incentive to stake ETH and become a validator. The network would be at risk without the presence of validators and the essential work they do.

## Conclusion

You successfully understand now what Solidity is and how smart contracts work. Furthermore, you've successfully built and deployed a smart contract.

## Resources
- [Wikipedia Smart Contract](https://en.wikipedia.org/wiki/Smart_contract)
- [Github Repository solidity](https://github.com/ethereum/solidity)
- [Solidity official document](https://docs.soliditylang.org/en/v0.8.19/)

_Article Photo by [bitpanda](https://www.bitpanda.com/academy/en/lessons/what-is-ethereum/)_


Build your first smart contract with Ethereum and Solidity

As part of Monstarlab's 3rd edition of the internal hackathon, [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), our team, Innovation Infusion, developed **The Safe Home Initiative for Children** which is a project designed to tackle the challenge faced by the partnering NGO [Ekmattra](https://ekmattra.org/).

The difficulty lies in analyzing the orphaned children's needs and finding the ideal match from a pool of organizations and schools that may not necessarily possess the required information to provide effective support.

This is where the Safe Home Initiative comes in, using advanced technologies like AI, blockchain, and machine learning to match children with the best organizations while ensuring their privacy and security.

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-The-problem-and-the-solution-Safe-Home/1.webp)

![]({{ sitePrefix }}/assets/img/articles/2023-05-25-The-problem-and-the-solution-Safe-Home/2.webp)

In this article, we outline the technologies used in building our solution.

## Backend technologies
- **Java**
- **Spring Boot**
- **AI Models**
- **Blockchain**
- **Data Security**

The backend of the **Safe Home Initiative** is built using Java and the Spring Boot framework, which allows for efficient data processing and management. AI models, such as ChatGPT-3.5, DeepFace, Facenet, and Facenet512, are used to gather information about children's interests, skills, and future plans. This information is then fed into a matching algorithm that utilizes blockchain technologies like Hyperledger Fabric and Corda, to guarantee the security and reliability of the system.

To ensure the privacy and security of the children's information, the Safe Home Initiative follows the OWASP and NIST Cybersecurity Framework. These frameworks provide guidelines and best practices to protect data from cyber threats, thereby safeguarding the children's information against unauthorized access and usage.

## Frontend technologies
- **JavaScript**
- **React**

The frontend of the Safe Home Initiative is built using JavaScript and the React framework, providing a user-friendly interface for NGOs and organizations to access and manage information about children and potential matches. This frontend interface facilitates the communication and evaluation of placements, thereby improving effectiveness and ensuring that children are matched with the most suitable organizations to meet their needs.

## Mobile technologies
- **Dart**
- **Kotlin**
- **Java**
- **Firebase**
- **Android**
- **Flutter**
- **iOS**
- **HealthKit**

The Safe Home Initiative also incorporates mobile technologies to monitor the health and wellbeing of the children. Mobile apps built using Dart, Kotlin, and Java, along with Firebase and HealthKit, facilitate easy tracking of the children's physical and emotional health. These apps are available on Android, Flutter, and iOS devices, making it easy for caregivers and organizations to monitor the children's progress and provide tailored support.

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-25-The-problem-and-the-solution-Safe-Home/diagram.webp>
<figcaption>User story</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-25-The-problem-and-the-solution-Safe-Home/architecture.webp>
<figcaption>Architecture</figcaption>
</figure>

## Conclusion

### Making a positive impact on orphaned children

Our team's MonstarHacks project combines advanced technologies with a commitment to social impact to revolutionize the matchmaking process for orphaned children.

By leveraging AI models, blockchain technologies, and machine learning algorithms, the Safe Home Initiative provides personalized matches and tailored support to help orphaned children succeed in the future.

With a focus on data security and privacy, the Safe Home Initiative offers a safe and reliable solution for NGOs like Ekmattra and other organizations to find the best matches for the children under their care.

The project's ultimate goal is to make a significant positive impact on the lives of orphaned children and provide them with the support they need to thrive.

_Article Photo by [Ekmattra](https://ekmattra.org/about-us/)_


The Safe Home Initiative for Children


[MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), the esteemed annual competition hosted by Monstarlab, has reached its 3rd edition, bringing together talented individuals from diverse backgrounds who are driven by innovation and dedicated to creating a positive social impact. This year, the theme was "NGOs" where 4 different organizations brought their own individual challenges for the hackathon participants to try to solve.

## Problem statement
One of the challenges put forward by [Ekmattra](https://ekmattra.org/), one of the partnering NGOs, draws attention to the pressing issue of children who are abandoned or who have fled from their villages. The task is to connect these children with the appropriate schools and NGOs that can offer them the necessary support for living and studying. The matchmaking process is considerably complex, often relying on impromptu decisions and relationships within a restricted network. This frequently sparks concerns about whether the decisions made truly serve the best interests of the children.

Recognizing the urgency of this challenge, our borderless team, known as `The CzechMate Coders`, selected this particular challenge to provide a solution that could have a lasting impact.

## The proposed solution
In this blog post, we will take you through our team's journey as we tackled the challenge presented by Ekmattra. We will discuss the reasons behind our choice of this specific challenge, the technical decisions we made, and the exceptional efforts we put forth during the competition. With collaboration, ingenuity, and a deep understanding of the problem at hand, our team aimed to make a lasting impact and provide vulnerable children with the support and opportunities they truly deserve.

We believe that a data-driven solution leveraging machine learning strategies can yield promising results. By identifying similarities and comparing profiles of children with those of potential partners, we can significantly simplify the process. To accomplish this, we employ a matching model that continuously learns and improves based on real data from actual cases. Given the wide variety of children's profiles, including factors such as age, health condition, social circumstances, special needs, and cultural-specific requirements, continuous training is crucial for the efficacy of the model.

Safety is our top priority, especially when it comes to children. We ensure that the system is not used for unethical activities or purposes. Moreover, we take into account the safety precautions implemented by partner sites when making the final choice. We maintain a system of regular feedback loops input by the NGO agent after each successful or unsuccessful case, with restricted access to the system to guarantee safe usage. This continuous training of the machine learning model ensures its reliability.

While our solution is digitalized, we strive to preserve the human factor, particularly in the final and most critical phase of the process. By combining technology with human intervention, we aim to create a comprehensive and effective matchmaking system for these vulnerable children.

## Our Tech stack
- Machine Learning
- React JS (Frontend)
- Python (Backend)

### Short presentation video
- [Demo](https://drive.google.com/file/d/1XkAZUCVquuK1x_ubMI4IMpcqENCsdtHT/view?usp=share_link)
- [Slides](https://docs.google.com/presentation/d/1-B_zK7ExYyKdKOn_ciBlUdQxsaIf1iAr57xjKBNnLBs/edit?usp=sharing)

### Attachments

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-23-Establishing-connections-for-children-living-on-the-streets-the-journey-of-The-CzechMate-Coders-in-MonstarHacks-2023/er-diagram.webp>
<figcaption>ER diagram</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-23-Establishing-connections-for-children-living-on-the-streets-the-journey-of-The-CzechMate-Coders-in-MonstarHacks-2023/ml-model.webp>
<figcaption>ML model</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-23-Establishing-connections-for-children-living-on-the-streets-the-journey-of-The-CzechMate-Coders-in-MonstarHacks-2023/homepage.webp>
<figcaption>Homepage</figcaption>
</figure>

<figure>
<img src={{ sitePrefix }}/assets/img/articles/2023-05-23-Establishing-connections-for-children-living-on-the-streets-the-journey-of-The-CzechMate-Coders-in-MonstarHacks-2023/match-page.webp>
<figcaption>Matches Page</figcaption>
</figure>

### Conclusion
The matchmaking system developed by The CzechMate Coders in response to Ekmattra's challenge has the potential to change the lives of vulnerable children living on the streets. The team's dedication, technical expertise, and innovation have enabled them to create a solution that can have a lasting impact and address a pressing social need. Thanks to Monstarlab, the MonstarHacks competition serves as a testament to the power of collaboration and ingenuity in creating positive social change. The impact of this project will undoubtedly be felt for years to come as children are given a chance for a better future.

_Article Photo by [Ekmattra](https://ekmattra.org/contact-us/)_


Establishing connections for children living on the streets, the journey of The CzechMate Coders in MonstarHacks 2023


With the growing need to build scalable, performing and maintainable enterprise services, microservices and event-driven architectures are often brought up as solutions to the problem, but what should you keep in mind when building services with such architectures?

## Introduction

This article explains the motivations for using Event-Driven Architecture, the core concepts, and its main challenges.
It also explains how to implement Event-Driven Architecture with NestJS as the backend and RabbitMQ as the communication tool to handle events and messaging.

## Microservices and their shortcomings

With the growing need for scalable, modular and maintainable systems, microservices architecture is becoming the de facto architecture for building complex enterprise systems.
With a microservices architecture, the system is broken down into smaller fault-tolerant modules that can be scaled, deployed, and maintained separately; individual services can even use different technologies stacks that might be more suitable to the domain or the task that the service is responsible for. For instance, a system where most services are mostly written in Java could be using Python for the service responsible for the Machine-Learning model.

One of the big advantages of going for a microservices architecture is having loosely coupled services so that a failure in one service does not cause failures in other services as a single point of failure. But that advantage can be lost if a blocking communication between services is causing it to become a single point of failure. This problem is very common when two services are communicating over HTTP/TCP in the request/response style.

We may also face latency problems as our service might be waiting for the other service to complete a long process to produce the response, causing slow waiting times that could be felt by the end user.

## Event-Driven Architecture

To solve this problem a non-blocking method of communication is needed, and that is where Event-Driven Architecture can solve those problems. With Event-Driven Architecture, instead of services communicating through a request/response method, **Event Producers** will post events to an **Event Queue** in the form of messages that will be consumed by **Event Consumers**. Different technologies will use different protocols to deliver their message and some will have support for streaming real-time data, but the basic architecture is as follows:

![Event-driven architecture diagram]({{ sitePrefix }}/assets/img/articles/2023-05-23-Introduction-To-Event-Driven-Architecture/eda-general.webp

Whenever an Event Producer produces an event, the event is registered in the Event Queue, the Event Producer then can start doing whatever other task it needs to do, as the Producer no longer cares about the response of that event. The event is now stored in the Queue and it can be consumed by the Event consumers that in turn will read the event message and do whatever processing is needed. When the necessary process is successfully finished, the message is deleted from the queue.

We can also have multiple instances of the Event Queue running, so we can guarantee that our Event Queue is fault-tolerable and that our services are now truly loose-coupled. Even in the event of one of our Consumers becoming unavailable, our Producers will still run as they should and the events will be stacked in our Queue, which can be picked up as soon as the needed Consumers are back up. This will also massively help with problems regarding data loss, as our Event Queue can have our messages saved for as long as we need, for as many retries as we need for the Consumer to process the event message.

## Event-Driven Architecture's main concepts

Here I would like to discuss some of the main concepts regarding Event-Driven Architecture:

- **Event**: A change in the status of an object or a resource. Eg: _A new order was made._
- **Event Producer**: A service that produces events to the Event Queue.
- **Event Queue/Broker**: A queue that holds event messages sent by Event Producers to be consumed by Event Consumers.
- **Event Consumer**: Consume events in the Event Queue and performs the necessary processing.
- **Dead Letter Queue**: A parallel queue to the main Event Queue where messages that for some reason could not be processed are moved to, so they can be analyzed and reprocessed if necessary at a later time. For instance, a message with some sort of corrupted information reaches that max number of retries and then is moved to the DLQ.
- **Ack**: Stands for acknowledgment; it acknowledges that an Event was successfully processed and that it can be safely removed from the queue.

## Event-Driven Architecture's challenges

Despite addressing many of the microservices architecture's problems, Event-Driven Architecture introduces its challenges.

### Complexity

To become able to develop well designed Event-Driven Architecture based enterprise system has a steep learning curve in both designing and implementing the architecture.

### Compatibility

Deciding if event-based communication is ideal for a pair or group of web services can be difficult. In some cases, there is no clear need for one service to be hanging waiting for another service to finish its processing. In other cases, the response from the consuming service process is utterly necessary to proceed with the task.

But sometimes it is not as clear if an event-based communication is the best answer for the services in question, and the trade-offs of different communication patterns must be weighed to decide which better suits the specific case.

### Event Duplication and Event Order

With queues running on multiple instances due to the fault-tolerant and horizontal-scaling nature of microservices, Event duplication is not a matter of if, but when. Dealing with event duplication is one of the big challenges of microservices, and it can be very dangerous depending on the kind of system you are developing. Eg: Billing your client twice, or adding funds to their wallet twice. Some message brokers will provide solutions to this out of the box, _with their own trade-offs_, and some will prefer to implement the deduplication logic themselves using event ids and other techniques.

On the same note, when event ordering matters it is important to implement a logic that guarantees that events are processed in the intended order. In systems where internet connection is rarer, a large number of events might be stacked before going to the server (Eg: IoT systems) and that could cause events to be all out of order if the logic to prevent it is not implemented.

### Error Handling

With more moving parts, different situations, different communication methods, etc. introduced by the Event-Driven Architecture, error handling becomes a lot more complex, but it is also a key part of fine-tuning your services.

## Event-Driven Architecture patterns

### Simple event processing

![Simple event processing]({{ sitePrefix }}/assets/img/articles/2023-05-23-Introduction-To-Event-Driven-Architecture/simple-event-processing.webp

### Complex Event Processing

![Complex event processing]({{ sitePrefix }}/assets/img/articles/2023-05-23-Introduction-To-Event-Driven-Architecture/complex-event-processing.webp

### Streaming Event Processing

![Streaming event processing]({{ sitePrefix }}/assets/img/articles/2023-05-23-Introduction-To-Event-Driven-Architecture/streaming-event-processing.webp

## When you SHOULD use Event-Driven Architecture

- When there is asynchronous processing between services or the current synchronous services can be remodeled to asynchronous with no major drawbacks.
- When the response from another service is not needed right away and the processing result can be managed by state/status transition. (Eg: Processing, Billing, Completed, Fail, etc).
- When process retries are needed and/or the failures need to avoid data loss.

## When you SHOULD NOT use Event-Driven Architecture

- The response from the service is integral for the process to continue. (Eg. Fraud detection)
- When the system is not complex enough for the benefits of an Event-Driven Architecture to outweigh the operational cost and complex implementation.
- The team is not used to Event-Driven Architecture and not enough time is available for the team to transition to the new architecture.

## Message Brokers and related technologies

- **RabbitMQ**: Works as a queue, pub/sub message broker using the AMQP protocol. Popular due to its simplicity and reliability.
- **Apache Kafka**: Distributed event store and streaming processing platform. One of the more popular options due to its performance and implementation possibilities.
- **AWS SQS/SNS, GCP Pub/sub**: Cloud-based message queue services that are ideal for those who want less operational overhead while working in the cloud.
- **AWS Kinesis**: A managed data streaming platform for AWS. Similar to Kafka, but with less operational overhead.

_Managed versions of RabbitMQ and Kafka can also be found in some cloud providers._

## Architecture example

A simple example of an ordering app using Event-Driven architecture:

![Ordering app architecture example]({{ sitePrefix }}/assets/img/articles/2023-05-23-Introduction-To-Event-Driven-Architecture/ordering-app-architecture.webp

With this architecture the Orders Service is decoupled from the Billing service, helping us achieve low latency to the end user, fault tolerance with the loosely coupled services that are able to perform retries on failure, and no data duplication since the Billing service does not need to have its own database.

On the other hand, the billing service is tightly coupled with the Fraud service, since any indication of fraud should cancel the Order right away. But even in the case of a failure in the Fraud service process, the billing service will only **Ack** the event message after the process is finished, so retries are also possible here thanks to the Event-Driven Architecture and Rabbit MQ.

## Code example

Here are a few code snippets of the main components of this architecture using the **NestJS** backend framework for **NodeJS** and **RabbitMQ**.

### Event Producer

`orders.controller.ts`:

```typescript
@Controller("orders")
export class OrdersController {
  constructor(private readonly ordersService: OrdersService) {}

  @Post()
  async createOrder(@Body() createOrderRequest: CreateOrderRequest) {
    return await this.ordersService.createOrder(createOrderRequest);
  }
}
```

`orders.service.ts`:

```typescript
@Injectable()
export class OrdersService {
  constructor(
    @InjectModel(Order.name) private orderModel: Model<OrderDocument>,
    @Inject(BILLING_SERVICE) private billingClient: ClientProxy
  ) {}

  async createOrder(
    createOrderRequest: CreateOrderRequest
  ): Promise<OrderInterface> {
    try {
      const order = new this.orderModel({
        ...createOrderRequest,
        status: "PENDING",
      });

      await order.save();
      const orderPayload: OrderInterface = {
        orderId: order._id.toString(),
        productName: order.productName,
        price: order.price,
        buyer: order.buyer,
        status: order.status,
      };

      await lastValueFrom(
        this.billingClient.emit("order_created", {
          orderPayload,
        })
      );

      return orderPayload;
    } catch (error) {
      throw error;
    }
  }
}
```

Our entry point is on our **controller** which will send the Order data to the **service** layer that will save the Order as **PENDING** and **emit** an **order_created** event to the **BILLING** queue with a payload, the queue is registered under the **BILLING_SERVICE** const variable. ( `export const BILLING_SERVICE = 'BILLING';`)

### Event Consumer

`billing.controller.ts`

```typescript
@Controller()
export class BillingController {
  constructor(
    private readonly billingService: BillingService,
    private readonly rmqService: RmqService,
    @Inject(ORDERS_SERVICE) private ordersClient: ClientProxy
  ) {}

  @EventPattern("order_created")
  async handleOrderCreated(
    @Payload("orderPayload") order: Order,
    @Ctx() context: RmqContext
  ) {
    //check for fraud
    if (await this.billingService.isFraud(order)) {
      order.status = "CANCELED_FRAUD";
    } else {
      this.billingService.bill(order);
      order.status = "PROCESSED";
    }

    // emit and event to update the order status
    await lastValueFrom(
      this.ordersClient.emit("order_procesed", {
        order,
      })
    );

    // if no errors happened, acknowledge the order_created process is over and remove it from the queue
    this.rmqService.ack(context);
  }
}
```

The Billing service now will be watching for the **Event Pattern** **_order_created_**, it will check for fraud and then try to bill the order. If no error occurs, it will then **emit** an **_order_processed_** event with the new order status. And if no errors happen it will then **Ack** the **_order_created_** event message.

The Order service will then update the database with the new status and proceed to **ack** the **order_processed** event message to end the whole process.

### Event Broker

`rmq.module.ts` (Configuration needed for services using RabbitMQ as Producers)

```typescript
@Module({
  providers: [RmqService],
  exports: [RmqService],
})
export class RmqModule {
  static register({ serviceName, serviceEnvVar }: RmqModuleOptions): DynamicModule {
    return {
      module: RmqModule,
      imports: [
        /*
          Creates a dynamic module based on the serviceName
          This module creates/connects to a queue as an event producer
        */
        ClientsModule.registerAsync([
          {
            name: serviceName,
            useFactory: (configService: ConfigService) => ({
              transport: Transport.RMQ,
              options: {
                urls: [configService.get<string>('RABBIT_MQ_URI')],
                queue: configService.get<string>(serviceEnvVar),
              },
            }),
            inject: [ConfigService],
          },
        ]),
      ],
      exports: [ClientsModule],
    };
}
```

The **register** method will be used by services that need to use RabbitMQ as **Producers**. The service will need to provide a **_serviceName_**, the **_URI for RabbitMQ_** (eg. running container), and **_serviceEnvVar_** as the environment variable holding the **_queue name_**.
For instance, you could have an environment variable `RABBIT_MQ_BILLING_QUEUE=BILLING`, so you would call the **_register_** method like `RmqModule.register({ serviceName: 'BILLING', serviceEnvVar: 'RABBIT_MQ_BILLING_QUEUE'}),`.

`rmq.service.ts`:

```typescript
@Injectable()
export class RmqService {
  constructor(private readonly configService: ConfigService) {}

  // return the config options to subscribe to a queue as an event consumer
  getOptions(queue: string, noAck = false): RmqOptions {
    return {
      transport: Transport.RMQ,
      options: {
        urls: [this.configService.get<string>("RABBIT_MQ_URI")],
        queue: this.configService.get<string>(queue),
        noAck,
        persistent: true,
      },
    };
  }
}
```

Returns the configuration needed for a service to consume from a RabbitMQ queue. (The queue variable is the env variable name).

### Event Message Ack

`rmq.service.ts`:

```typescript
@Injectable()
export class RmqService {
  constructor(private readonly configService: ConfigService) {}

  ack(context: RmqContext): void {
    const channel = context.getChannelRef();
    const originalMessage = context.getMessage();
    channel.ack(originalMessage);
  }
}
```

The method needed to **Ack** event messages from a queue when they are no longer needed.

## References

- The complete code for the above architecture can be found in the [following Github repository](https://github.com/FelipeCO14/nest-event-driven-app)
- [NestJS + RabbitMQ integration](https://docs.nestjs.com/microservices/rabbitmq)
- [IBM Event-Driven Architecture article](https://www.ibm.com/topics/event-driven-architecture)

_Article Photo by [Krusche & Company](https://kruschecompany.com/)_


An Introduction to Event-Driven Architecture


Dealing with families with a seriously ill child can be an emotionally challenging experience for them and requires the utmost sensitivity. Currently, the admissions process has its own challenges. The process involves the manual transfer of personal details and information from cards to an Excel spreadsheet, which is then transferred to their internal CRM system. This process is not only time-consuming but also prone to human errors, resulting in inaccurate records. Additionally, there are risks to the security of personal data due to the storage of cards in a secure box or filing cabinet.

To address these challenges and improve the admissions process, our Team Monstars Inc., as part of the [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks) 2023 Monstarlab internal hackathon, proposed a value-added solution for the NGO [The Sick Children's Trust](https://www.sickchildrenstrust.org/) delivered in three phases (short-term, medium-term, and long-term) through the development of a super app. The short-term solution will consist of two steps. The first step will address the first pain point in the data collection process by implementing an API integration between hospitals partnered with the NGO and the CRM tools currently in place by the NGO for automatic data transfer.

The second step will involve configuring integration between the NGO CRM and ChatGPT image analysis capabilities to extract the relevant data into the CRM system automatically. This two-step solution will provide immediate value creation for the NGO, allowing the team to focus on developing plans to achieve the long-term vision of a super app called **weCare**. Our plan will deliver benefits in the short, medium, and long term for both the NGO and the families they help.

![]({{ sitePrefix }}/assets/img/articles/2023-05-22-Improving-the-admissions-process-for-seriously-ill-children-and-their-families/design.webp)

## Short-term solution

We propose the creation of an API between hospitals and the NGO CRM to mitigate the risks of inaccurate records and data security. The API will retrieve patient data upon consent directly from the hospital's electronic medical records system. This will save time and effort while ensuring the accuracy of data. Additionally, this API will comply with GDPR regulations, ensuring the security and privacy of personal data.

By digitizing this process, we can eliminate human errors and ensure the security of personal data. This resolves the most pressing challenge of manual information collection with creative and simple digital solutions. We will also create an integration between ChatGPT image upload capabilities and the NGO CRM to automatically extract the relevant data from the physically stored cards directly into the CRM tool.

## Medium-term solution

To further improve the admissions process, we propose the creation of an app that integrates the various functions of the NGO. The app will provide staff with a unified platform to manage guests' and partners' data. It will be designed with a user-friendly interface to ensure ease of use for both staff and families. The app will provide automated emotional support services and guides to the facilities and local points of interest.

## Long-term Solution

In the longer perspective, the super app **weCare** will be gradually updated to further support the NGO and their guest experience to provide 360-degree support through automated processes. the NGO will have access to a larger set of guest data, which can be used to maintain and build on their existing guest relationship management. This can include automated notification of when it's a good time to reach out to previous guests for fundraising or other requests.

## Conclusion

Improving the admissions process for families with seriously ill children is a crucial step towards ensuring that they receive the care and support they need. By implementing an API between hospitals and the NGO, along with the creation of a super app, we can streamline the admissions process, eliminate human errors, ensure data security, and improve the overall guest experience.

Our immediate solution will allow for a modular and scalable process that streamlines admissions and data collection, by replacing the manual card system with a digital solution. In the medium term, we propose the creation of the app weCare. In the long term, we propose the expansion of the created app into a super app to integrate the various functions of the NGO.

With these solutions, we can ensure that families feel welcome and supported during their stay, allowing the NGO to really focus on what matters - supporting the families.

_Article Photo by [The Sick Children's Trust](https://www.sickchildrenstrust.org/)_


Improving the admissions process for seriously ill children and their families


Participating in [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks), Monstarlab's yearly internal hackathon, for the first time was an experience outside our everyday project routine. Below, we share our experience and provide insight into the product we aimed to create.

## Why was this challenge interesting to us?

NGOs play an important role in addressing social and environmental challenges around the world. They often work with limited resources and staff, and technology can help them to increase their impact and reach more people in need. As engineers, our role in society is to utilize our technical skills and knowledge for its betterment.

## Our team, Team Rakshak:

- Shaikh Huma: Flutter developer from the Monstarlab Tokyo office
- Dinakar Maurya: Lead developer from the Monstarlab Tokyo office

## Deciding on the topic

On Day 1 of the hackathon, we opted for a morning call to decide on the NGO with which we would collaborate and to brainstorm ideas. We chose to proceed with [Asha](https://www.asha-np.org/en), an NGO based in Japan. Given that both team members were from the Tokyo office, we were eager to take up this challenge. Upon reading the problem statement and brainstorming a few ideas, we came across several challenges. We addressed these in discussions with our mentor. Subsequently, we decided to have a conversation with the NGO representative, thus scheduling a meeting with him on Day 2. After getting more insights on the problem statement, we moved forward with our product prototyping.

## Working on the product

The hackathon spanned across 3 days, coinciding with Japan's Golden Week, which is typically a vacation period. Though we were in a holiday mood, we still went forward with all enthusiasm to build our product. Since we had no designers on the team, we sketched out a basic app structure with our features and began working on the app, which was coded in Flutter. With Dinakar leading the team, tasks were allocated, and we commenced work, holding meetings throughout the three days.

## The problem statement

- **How do we deliver practical training sessions (Injury Assessment, First-Aid Kit)?**
  Digital Solution - We can deliver smooth and fast video lectures using the Agora SDK.
- **How do we overcome the human resources barrier (limited lecturers) to provide lectures in each school?**
  We solve this issue by offering live lecture sessions and using auto-recording for future repeated lectures.
- **How do we make it attractive to local people?**
  We have an easy-to-use app with a single screen. The app includes a feature powered by Chat GPT, which can answer all kinds of questions in the local language.
- **How do we overcome the financial resource barrier?**
  We use a single codebase Flutter app that runs on all platforms, including iOS and Android.
- **How do we overcome the language barrier when using the system (laptop, mobile phones, iPad)?**
  We provide solutions in four languages. We can offer support for multiple languages.
- **Can the app be used offline if there is no internet?**
Yes, the app can be used both online and offline for the convenience of the users.

## Participating for the first time in MonstarHacks

We participated in a hackathon for the first time, which was both a thrilling and challenging experience. It gave us a unique opportunity to collaborate with individuals from diverse backgrounds and work on a problem or project within a specific timeframe. As someone who has never been a part of a hackathon before, the experience was daunting, yet exhilarating.

At first, we felt overwhelmed by the fast-paced environment of a hackathon and the number of teams taking part from other offices. Other participants were bustling around, working on their projects with intense focus, which made us wonder if we could keep up and make a meaningful contribution.

As we started to work on the project, we felt a mix of emotions. On one hand, there was a sense of excitement and enthusiasm about the project and the opportunity to learn and collaborate with others. On the other hand, there was a feeling of anxiety or pressure to perform well and deliver a successful project within the time limit.

While working on our project, we encountered challenges and roadblocks. Despite these obstacles, there was a sense of camaraderie within the team. We were able to bounce ideas off each other, share knowledge and expertise, and work towards a common goal.

As the hackathon came to a close, we felt a sense of pride in what we had accomplished, yet were nervous about presenting our project to the judges and other participants. Regardless, the experience was incredibly rewarding, providing valuable lessons and insights. Whether we won or lost, we gained invaluable experience and skills that can be applied to future projects and endeavors.

## The proposed solution

- Our solution is a Flutter based app.
- **Volunteer Login** - To be used in middle schools with the following features:
  - Login
  - Video list
  - See the past lectures list
  - Play lecture from the list
  - Chat/ Help
  - Join a live session
- **Lecturer Login** - To be used for recording lectures with the following features:
  - Login
  - Video list
  - Chat/ Help
  - Host a live session
  - Record a lecture
  - Record a lecture using video screen
  - Preview and save lecture

## App flow

![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/app-flow.webp)
![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/app-flow-volunteer.webp)
![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/app-flow-lecturer.webp)

## App UI

![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/screen-en.webp)
![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/screen-jp.webp)
![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/screen-hi.webp)
![]({{ sitePrefix }}/assets/img/articles/2023-05-17-Team-Rakshaks-experience-taking-part-in-MonstarHacks-2023/screen-ne.webp)

## Future scope proposed

- Voice support through AI Chat GPT
- Capability to find local professionals
- Addition of FAQ, Contact Us, and other informative pages
- Tutorial screens that appear immediately after login
- Ability to answer custom NGO questions
- Features related to education and teaching methods
- A reference blog for additional information
- Synchronization of local videos with remote videos
- Chat feature during video calls for a more interactive live session

*Please note that these features are not yet available and may be implemented at a later time.*

## Costs

- Expense related to the necessary hardware such as mobile phones, Android phones, Android tablets, iPhones, and iPads.
- Cost involved in local video recording such as server storage, local mobile phone storage, and free Google Cloud Hosting for backend server/cloud hosting.
- Charges for using Agora video calls, with a pricing model of the first 10,000 minutes free every month, equivalent to approximately 166 hours. For further details, refer to their website https://www.agora.io/en/pricing/.

## Our tech stack

- **ChatGPT**, available at https://openai.com/, offers an advanced chat engine that uses AI to enhance work and creativity.
- **Agora Video Call**, accessible at https://videocall.agora.io/welcome, provides free HD video and voice calls that can be made anytime and anywhere.
- **Flutter**, a framework created by Google and accessible at https://docs.flutter.dev/get-started/install, is an open-source tool that enables building multi-platform, natively compiled, and visually stunning applications from a single codebase.
- **Dart**, programming language available at https://dart.dev/, is a language optimized for clients that enables fast app development on multiple platforms.

## Conclusion

In summary, we crafted a small and functioning prototype to enhance health literacy for Asha NGO as part of MonstarHacks.

## Related Articles

- [ChatGPT meets Flutter: A step by step integration guide - English](https://engineering.monstar-lab.com/en/post/2023/05/13/ChatGPT-meets-Flutter-A-step-by-step-integration-guide/)
- [FlutterでのChatGPTの統合 - Japanese](https://engineering.monstar-lab.com/jp/post/2023/05/08/Integration-of-ChatGPT-in-Flutter/)

## References

- [MonstarLab Flutter Template - OSS Repository](https://github.com/monstar-lab-oss/flutter-template-riverpod)


Team Rakshak's experience taking part in MonstarHacks 2023


When it comes to securing web applications, there are various options for authentication and authorization. Two popular choices are PASETO and JWT.

While JWT has been widely utilized for many years, PASETO is a relatively new technology that has gained traction due to its enhanced security features.

PASETO (Platform-Agnostic Security Tokens) is both a specification and a reference implementation for secure stateless tokens. It serves as a highly secure alternative to JWT.

### Understanding Token-Based Authentication

The typical flow for authenticating a user in a secured API involves the following steps:

1. The user provides their username and password for authentication.
2. Upon successful verification, the API returns an access token (either JWT or PASETO).
3. The access token is included in the Authorization header when making requests to protected endpoints.
4. The API server validates the token and responds with the appropriate secured data if the access is valid.

![Modern Token Base Authentication]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/token_based_authentication.webp)

### Understanding JWT and Its Limitations

JWT consists of three parts:

- The header, which contains the token's signing algorithm.
- The payload, which holds information about the authenticated user and additional data. The server can customize this part of the payload.
- The signature, generated by the server using a private key. This signature enables the server to verify the authenticity of the JWT during the validation process.

![JWT Demo]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/jwt.webp

_Photo by [Wallarm](https://www.wallarm.com/what/token-based-authentication)_

While JWT provides flexibility in choosing the digital signature algorithm and verification implementation, this flexibility also introduces vulnerabilities.

There are multiple algorithms to choose from, some of which may be weak and susceptible to attacks, such as ECDSA (vulnerable to invalid-curve attacks) or RSA PKCSv1.5 (vulnerable to padding oracle attacks).

JWT implementations are also prone to errors, which can result in security vulnerabilities like broken JWT validation.

When used correctly, JWT can be a reliable and flexible authentication system. However, caution must be exercised to avoid exposing the server to potential attacks.

In contrast, PASETO addresses these issues by simplifying the implementation process.

## PASETO: The Solution

While JWT offers implementation flexibility, PASETO takes a more rigid approach. However, this rigidity helps prevent implementation errors and misuse.

PASETO is designed to be user-friendly and offers higher cryptographic resilience compared to JWT.

When using PASETO, the user only needs to configure two settings:

- The PASETO version (**`v1`**, **`v2`**, **`v3`**, or **`v4`**) specified via the **`version`** field of the token.
- Whether encryption and decryption should be symmetric or asymmetric, indicated by the **`purpose`** field of the token.

That's all it takes.

### PASETO Token Structure

Similar to JSON Web Tokens (JWTs), PASETO tokens are composed of dot-separated base64url encoded data organized in the following format:

```
version.purpose.payload.footer
```

- **`version`**: Allows for incremental improvements to the token format. The current versions are "v1," "v2," "v3," and "v4."
  - **`v1`**: Utilizes strong cryptographic primitives that are widely available today.
  - **`v2`**: Utilizes newer and stronger cryptographic primitives, but is supported by fewer cryptographic libraries.
- **`purpose`**: A concise string that describes the token format as either "local" or "public."
  - **`local`**: The token's payload is encrypted and can only be accessed by parties possessing the shared key.
  - **`public`**: The payload is NOT encrypted; instead, it is signed and verified using a public key.
- **`payload`**: Encoded data with a format specific to the token's version and purpose.
- **`footer`** (optional): Unencrypted JSON, typically used to store the ID of a public key for token validation.

All PASETO token formats are tamperproof, ensuring that any modifications to the token result in failed validation.

### Local Tokens (Symmetric Encryption)

Local tokens are always symmetrically encrypted using a shared secret key. This means that the contents of a local PASETO token cannot be viewed without the correct secret key.

![Symmetric encryption]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/symmetric.webp

Here is an example of a local PASETO token, including its decoded payload, optional footer, and the signing key used to sign the information.

![Symmetric encryption]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/paseto_symmetric_demo.webp)

### Public Tokens (Asymmetric Encryption)

Public PASETO tokens are suitable for scenarios where it is not safe to share a secret key with all involved parties.

Public tokens are not encrypted but are digitally signed. This means that if an attacker obtains a public PASETO token, they can view its contents but cannot modify it without detection due to the digital signatures used in PASETO tokens.

If an attempt is made to verify a maliciously modified public PASETO token, an error will occur.

![Asymmetric encryption]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/asymmetric.webp

Here is an example of a public PASETO token, including its decoded payload, optional footer, and the public and private keys used to sign the information.

![Asymmetric encryption]({{ sitePrefix }}/assets/img/articles/2023-05-16-An-overview-of-PASETO-Token-Based-Authentication/paseto_asymmetric_demo.webp)

### Versions

Each PASETO version introduces improvements over its predecessor. To implement the PASETO specification correctly, refer to the details provided for each version:

[https://github.com/paseto-standard/paseto-spec/tree/master/docs/01-Protocol-Versions](https://github.com/paseto-standard/paseto-spec/tree/master/docs/01-Protocol-Versions)

### Libraries

You can find all the libraries implementing PASETO for all the popular languages along with the supported versions here:

[https://paseto.io/](https://paseto.io/)

### Implementation using the Ruby library

We will follow an example of usage of the Ruby library implementing PASETO:

[https://github.com/bannable/paseto](https://github.com/bannable/paseto)

We need to first install the gem.

```yaml
gem 'ruby-paseto'
gem 'rbnacl', '~> 7.1.1' # optional - only if PASETO version 4 will be used
```

### Symmetric encryption (local)

```ruby
require 'paseto'

####################
#### ENCRYPTION ####
####################

# typically, this shared 32 bytes key is stored in both the authentication server
# and the client server
shared_secret_key = SecureRandom.bytes(32)

# initialize the PASETO encrypter/decrypter
crypt = Paseto::V4::Local.new(ikm: shared_secret_key) # version: v4 / purpose: local

# payload in plain text
claims = { "company" => "monstarlab" }
footer = { "viewable" => "yes" }

# encode the payload and get the PASETO
encrypted_token = crypt.encode(claims, footer: JSON.dump(footer))
# => "v4.local.E1Y_KQ6Ek8lSOKrJ6kI1YjWXfAKJ0OEcdhUPywznjBjK5SGDUr4-6rbaZk-CIM_mdQgQHGPj8yAWQswktkCe_Sm_Nj9eEfDxNBFeAC2KsgqFCjF07VJo5ail0jnSTNM0-PekMytJleea8OvNkKdoLs4GKAsZTTJ_-DEMmOMyVlddlmaWoVwnF2JkpjBzFRO7d6PlIIY29rQWOXSvZxoLEqkE5XJvHpFs4NTuCHnF4Pko10X_sgHCPkTGXkWNDg.eyJ2aWV3YWJsZSI6InllcyJ9"

####################
#### DECRYPTION ####
####################

# typically, this shared 32 bytes key is stored in both the authentication server
# and the client server
shared_secret_key = SecureRandom.bytes(32)

# initialize the PASETO encrypter/decrypter
crypt = Paseto::V4::Local.new(ikm: shared_secret_key) # version: v4 / purpose: local
encrypted_token = "v4.local.E1Y_KQ6Ek8lSOKrJ6kI1YjWXfAKJ0OEcdhUPywznjBjK5SGDUr4-6rbaZk-CIM_mdQgQHGPj8yAWQswktkCe_Sm_Nj9eEfDxNBFeAC2KsgqFCjF07VJo5ail0jnSTNM0-PekMytJleea8OvNkKdoLs4GKAsZTTJ_-DEMmOMyVlddlmaWoVwnF2JkpjBzFRO7d6PlIIY29rQWOXSvZxoLEqkE5XJvHpFs4NTuCHnF4Pko10X_sgHCPkTGXkWNDg.eyJ2aWV3YWJsZSI6InllcyJ9"

# note that the last part of the token `eyJ2aWV3YWJsZSI6InllcyJ9` is only a base64 encoded string
# meaning anyone can see its contents
Base64.decode64("eyJ2aWV3YWJsZSI6InllcyJ9")
# => "{\"viewable\":\"yes\"}"

# decoding the token and get the payload
crypt.decode(encrypted_token)
# => <Paseto::Result
#           claims={
#              "exp"=>"2023-05-10T11:18:41+09:00",
#              "iat"=>"2023-05-10T10:18:41+09:00",
#              "nbf"=>"2023-05-10T10:18:41+09:00",
#              "company"=>"monstarlab"},
#           footer={"viewable"=>"yes"}
#    >

# if the token has been maliciously modified, an error will be raised
encrypted_token[-1] = "M"
crypt.decode(encrypted_token)
# Paseto::InvalidAuthenticator: Paseto::InvalidAuthenticator
# from /Users/tony_duong/.rvm/gems/ruby-3.1.3/gems/ruby-paseto-0.1.2/lib/paseto/symmetric_key.rb:53:in `decrypt'

```

### Usage of asymmetric encryption (public)

We first start by generating a public/private key pair.

```bash
ssh-keygen
# Output: public key and private key
```

```ruby
require 'paseto'

####################
#### ENCRYPTION ####
####################

# initialize the PASETO encrypter/decrypter
pem = File.read('private_key')
signer = Paseto::V4::Public.new(pem)

# payload in plain text
claims = { "company" => "monstarlab" }
footer = { "viewable" => "yes" }

# encode the payload and get the PASETO
signed_token = signer.encode(claims, footer: footer)
# => "v4.public.eyJleHAiOiIyMDIzLTA1LTEwVDExOjQ2OjA0KzA5OjAwIiwiaWF0IjoiMjAyMy0wNS0xMFQxMDo0NjowNCswOTowMCIsIm5iZiI6IjIwMjMtMDUtMTBUMTA6NDY6MDQrMDk6MDAiLCJjb21wYW55IjoibW9uc3RhcmxhYiJ9taKQPCARAZHv85xk7yaWPDeWHaHt981eHmoiYIrIcA-monnIbMax2EDxIjObgr6qLLuYzAH4BK5N6q0TJANeBg.eyJ2aWV3YWJsZSI6InllcyJ9"

####################
#### DECRYPTION ####
####################

verifier = Paseto::V4::Public.new('public_key')

# when initialized with a public key, only verification/decoding can be performed
# if encode is called, error is raised
verifier.encode({'foo' => 'bar'})
# => ArgumentError

signed_token = "v4.public.eyJleHAiOiIyMDIzLTA1LTEwVDExOjQ2OjA0KzA5OjAwIiwiaWF0IjoiMjAyMy0wNS0xMFQxMDo0NjowNCswOTowMCIsIm5iZiI6IjIwMjMtMDUtMTBUMTA6NDY6MDQrMDk6MDAiLCJjb21wYW55IjoibW9uc3RhcmxhYiJ9taKQPCARAZHv85xk7yaWPDeWHaHt981eHmoiYIrIcA-monnIbMax2EDxIjObgr6qLLuYzAH4BK5N6q0TJANeBg.eyJ2aWV3YWJsZSI6InllcyJ9"
verifier.decode(signed_token)
# => <Paseto::Result
#           claims={
#              "exp"=>"2023-05-10T11:18:41+09:00",
#              "iat"=>"2023-05-10T10:18:41+09:00",
#              "nbf"=>"2023-05-10T10:18:41+09:00",
#              "company"=>"monstarlab"},
#           footer={"viewable"=>"yes"}
#    >
```

## Conclusion

In conclusion, we have explored the vulnerabilities that can arise from the careless use of JSON Web Tokens (JWTs). While JWTs can serve as an effective means of incorporating authentication into a system when used correctly, their flexible specification can potentially lead to implementation errors and subsequent security issues.

To address these concerns, PASETO has been introduced as an alternative solution with specific design goals in mind:

- Simplicity of use: PASETO simplifies the token creation process by requiring only the specification of the **`purpose`** and **`version`** parameters.
- Resistance to implementation errors: Unlike JWTs, PASETO eliminates the need to choose from a wide range of potentially insecure cryptographic algorithms, thereby reducing the risk of implementation mistakes.

PASETO takes a developer-first approach to security tokens by streamlining the decision-making process for developers. By offering two distinct purposes, namely the choice between a symmetric or asymmetric security model, PASETO automatically selects the most suitable options for authenticated encryption and digital signatures. This ensures that your tokens remain secure and immune to cryptographic vulnerabilities.

Overall, PASETO provides a more robust and straightforward approach to security token management, mitigating the risks associated with JWTs while maintaining a high level of security for your system.

## References

- [PASETO: The JWT Killer? - article by Sandesh Dahake](https://www.linkedin.com/pulse/paseto-jwt-killer-sandesh-dahake/)
- [PASETO: Platform-Agnostic Security Tokens - Github repository](https://github.com/paseto-standard/paseto-spec)
- [Why PASETO is better than JWT for token-based authentication? - Youtube video by Tech School](https://www.youtube.com/watch?v=nBGx-q52KAY&ab_channel=TECHSCHOOL)
- [Introducing JPaseto: Security Tokens For Java - article by Brian Demers](https://developer.okta.com/blog/2020/07/23/introducing-jpaseto)
- [Encode or Decode PASETO](https://token.dev/paseto/)
- [Paseto - Github repository](https://github.com/bannable/paseto)

_Article Photo by [Erik Mclean](https://unsplash.com/photos/XUJcmgEhpjA)_


An overview of PASETO Token-Based Authentication


We had a blast participating for the first time in [MonstarHacks](https://www.linkedin.com/showcase/monstarhacks) and even though the pressure can be very high due to the limited amount of time, the resulting ideas can be truly amazing. Here we would like to share a little bit of our discoveries and solution trying to solve the [El Gatio](https://elgatio.org/) NGO challenge.

![Monstarlab colombian coders]({{ sitePrefix }}/assets/img/articles/2023-05-15-Monstarhacks-solution-to-decrease-pet-relinquishment-in-a-colombian-NGO-shelter/chicharron-coders.webp

### The problem

Overpopulation of stray pets is a complex problem that has far-reaching environmental consequences that should not be ignored. Stray pets can disrupt natural food chains, hunt and kill native wildlife, and spread disease to other animals in the area. But yet there's very limited governmental resources being directed to address this problem in a humane way. This is why cat rescue organizations face some big challenges as they are obligated to work with their own resources making it very difficult and time-consuming to find foster homes, match cats with the right families to avoid future relinquishments and perform a continuous follow-up process.

### Insights we found

1. After some investigation we found that the common approach for shelters is to treat foster home applications and adoption application as two different processes, but for us is actually a single one, because people's intention may vary during time and what first started as a fostering can evolve into an adoption.

2. Creating matches for successful adoptions (aka adoption without relinquishment) is much more complex than just measuring compatibility between two profiles. Because this is more like finding the “love of your life” and chemistry resulting from physical interaction plays a substantial role into determining whether you’ll commit to a pet or not.

**Our conclusion**: If we allow people to live with cats for a trial period in a foster home, we can likely focus on converting positive experiences into successful adoptions, and prevent adoptions from happening when people are not ready to commit.

### Our Solution
Transform the shelter's fostering and adoption processes into a unified system by implementing SaaS (Software as a Service) trial best practices, with systematic follow-up checks to encourage adoption, enhance post-adoption retention, and expand the network of foster homes. Therefore:

- People who only want to be foster homes (for their own reasons) could remain in a "trial version" with different cats indefinitely.
- People who wanted to be foster homes and, during the "trial," develop a good relationship with the cat could be led/converted into adopting the cat (aka "upgrading from the trial version").
- People who want to adopt but are inexperienced may start to develop a poor relationship with the cat. In such cases, they could receive priority assistance to allow the shelter to provide guidance and improve the engagement.
- People who want to adopt and, during the "trial," start developing a bad relationship with the cat, despite assistance, could have the option to trial a different cat or decide not to adopt if they realize they aren't prepared.
- People who want to adopt and, during the "trial," develop a good relationship with the cat will likely cement their decision and become the most loyal adopters.

This will enable us to solve the most complex problems of the challenge: increasing the number of available foster homes while at the same time reducing the number of people who would ultimately return an adopted cat. By incorporating systematic follow-ups, the shelters will be able to conserve resources with users who are managing well, while also providing timely guidance to those in need. This approach will also help identify opportunities for fostering to adoption conversion. Most importantly, it will allow us to understand and identify friction points in the cat-owner relationships and improve them for the benefit of people's and pets' well-being and harmony in our society.

![Applicant's app]({{ sitePrefix }}/assets/img/articles/2023-05-15-Monstarhacks-solution-to-decrease-pet-relinquishment-in-a-colombian-NGO-shelter/app.webp

![Shelter's backoffice]({{ sitePrefix }}/assets/img/articles/2023-05-15-Monstarhacks-solution-to-decrease-pet-relinquishment-in-a-colombian-NGO-shelter/backoffice.webp

### Resources

- [The impact of returning a pet to the shelter on future animal adoptions](https://www.nature.com/articles/s41598-022-05101-5)
- [What Do People Want? Factors People Consider When Acquiring Dogs, the Complexity of the Choices They Make, and Implications for Nonhuman Animal Relocation Programs](https://www.tandfonline.com/doi/abs/10.1080/10888705.2014.943836)
- [El Gatio shelter](https://elgatio.org/)

_Article Photo by [El Gatio](https://elgatio.org/)_
