Coding Resources

DZone's Featured Coding Resources

Development at Scale

As organizations’ needs and requirements evolve, it’s critical for development to meet these demands at scale. The various realms in which mobile, web, and low-code applications are built continue to fluctuate. This Trend Report will further explore these development trends and how they relate to scalability within organizations, highlighting application challenges, code, and more.

Spring Strategy Pattern Example

By Max Stepovyi

In this example, we'll learn about the Strategy pattern in Spring. We'll cover different ways to inject strategies, starting from a simple list-based approach to a more efficient map-based method. To illustrate the concept, we'll use the three Unforgivable curses from the Harry Potter series — Avada Kedavra, Crucio, and Imperio. What Is the Strategy Pattern? The Strategy Pattern is a design principle that allows you to switch between different algorithms or behaviors at runtime. It helps make your code flexible and adaptable by allowing you to plug in different strategies without changing the core logic of your application. This approach is useful in scenarios where you have different implementations for a specific task of functionality and want to make your system more adaptable to changes. It promotes a more modular code structure by separating the algorithmic details from the main logic of your application. Step 1: Implementing Strategy Picture yourself as a dark wizard who strives to master the power of Unforgivable curses with Spring. Our mission is to implement all three curses — Avada Kedavra, Crucio and Imperio. After that, we will switch between curses (strategies) in runtime. Let's start with our strategy interface: Java public interface CurseStrategy { String useCurse(); String curseName(); } In the next step, we need to implement all Unforgivable curses: Java @Component public class CruciatusCurseStrategy implements CurseStrategy { @Override public String useCurse() { return "Attack with Crucio!"; } @Override public String curseName() { return "Crucio"; } } @Component public class ImperiusCurseStrategy implements CurseStrategy { @Override public String useCurse() { return "Attack with Imperio!"; } @Override public String curseName() { return "Imperio"; } } @Component public class KillingCurseStrategy implements CurseStrategy { @Override public String useCurse() { return "Attack with Avada Kedavra!"; } @Override public String curseName() { return "Avada Kedavra"; } } Step 2: Inject Curses as List Spring brings a touch of magic that allows us to inject multiple implementations of an interface as a List so we can use it to inject strategies and switch between them. But let's first create the foundation: Wizard interface. Java public interface Wizard { String castCurse(String name); } And we can inject our curses (strategies) into the Wizard and filter the desired one. Java @Service public class DarkArtsWizard implements Wizard { private final List<CurseStrategy> curses; public DarkArtsListWizard(List<CurseStrategy> curses) { this.curses = curses; } @Override public String castCurse(String name) { return curses.stream() .filter(s -> name.equals(s.curseName())) .findFirst() .orElseThrow(UnsupportedCurseException::new) .useCurse(); } } UnsupportedCurseException is also created if the requested curse does not exist. Java public class UnsupportedCurseException extends RuntimeException { } And we can verify that curse casting is working: Java @SpringBootTest class DarkArtsWizardTest { @Autowired private DarkArtsWizard wizard; @Test public void castCurseCrucio() { assertEquals("Attack with Crucio!", wizard.castCurse("Crucio")); } @Test public void castCurseImperio() { assertEquals("Attack with Imperio!", wizard.castCurse("Imperio")); } @Test public void castCurseAvadaKedavra() { assertEquals("Attack with Avada Kedavra!", wizard.castCurse("Avada Kedavra")); } @Test public void castCurseExpelliarmus() { assertThrows(UnsupportedCurseException.class, () -> wizard.castCurse("Abrakadabra")); } } Another popular approach is to define the canUse method instead of curseName. This will return boolean and allows us to use more complex filtering like: Java public interface CurseStrategy { String useCurse(); boolean canUse(String name, String wizardType); } @Component public class CruciatusCurseStrategy implements CurseStrategy { @Override public String useCurse() { return "Attack with Crucio!"; } @Override public boolean canUse(String name, String wizardType) { return "Crucio".equals(name) && "Dark".equals(wizardType); } } @Service public class DarkArtstWizard implements Wizard { private final List<CurseStrategy> curses; public DarkArtsListWizard(List<CurseStrategy> curses) { this.curses = curses; } @Override public String castCurse(String name) { return curses.stream() .filter(s -> s.canUse(name, "Dark"))) .findFirst() .orElseThrow(UnsupportedCurseException::new) .useCurse(); } } Pros: Easy to implement. Cons: Runs through a loop every time, which can lead to slower execution times and increased processing overhead. Step 3: Inject Strategies as Map We can easily address the cons from the previous section. Spring lets us inject a Map with bean names and instances. It simplifies the code and improves its efficiency. Java @Service public class DarkArtsWizard implements Wizard { private final Map<String, CurseStrategy> curses; public DarkArtsMapWizard(Map<String, CurseStrategy> curses) { this.curses = curses; } @Override public String castCurse(String name) { CurseStrategy curse = curses.get(name); if (curse == null) { throw new UnsupportedCurseException(); } return curse.useCurse(); } } This approach has a downside: Spring injects the bean name as the key for the Map, so strategy names are the same as the bean names like cruciatusCurseStrategy. This dependency on Spring's internal bean names might cause problems if Spring's code or our class names change without notice. Let's check that we're still capable of casting those curses: Java @SpringBootTest class DarkArtsWizardTest { @Autowired private DarkArtsWizard wizard; @Test public void castCurseCrucio() { assertEquals("Attack with Crucio!", wizard.castCurse("cruciatusCurseStrategy")); } @Test public void castCurseImperio() { assertEquals("Attack with Imperio!", wizard.castCurse("imperiusCurseStrategy")); } @Test public void castCurseAvadaKedavra() { assertEquals("Attack with Avada Kedavra!", wizard.castCurse("killingCurseStrategy")); } @Test public void castCurseExpelliarmus() { assertThrows(UnsupportedCurseException.class, () -> wizard.castCurse("Crucio")); } } Pros: No loops. Cons: Dependency on bean names, which makes the code less maintainable and more prone to errors if names are changed or refactored. Step 4: Inject List and Convert to Map Cons of Map injection can be easily eliminated if we inject List and convert it to Map: Java @Service public class DarkArtsWizard implements Wizard { private final Map<String, CurseStrategy> curses; public DarkArtsMapWizard(List<CurseStrategy> curses) { this.curses = curses.stream() .collect(Collectors.toMap(CurseStrategy::curseName, Function.identity())); } @Override public String castCurse(String name) { CurseStrategy curse = curses.get(name); if (curse == null) { throw new UnsupportedCurseException(); } return curse.useCurse(); } } With this approach, we can move back to use curseName instead of Spring's bean names for Map keys (strategy names). Step 5: @Autowire in Interface Spring supports autowiring into methods. The simple example of autowiring into methods is through setter injection. This feature allows us to use @Autowired in a default method of an interface so we can register each CurseStrategy in the Wizard interface without needing to implement a registration method in every strategy implementation. Let's update the Wizard interface by adding a registerCurse method: Java public interface Wizard { String castCurse(String name); void registerCurse(String curseName, CurseStrategy curse) } This is the Wizard implementation: Java @Service public class DarkArtsWizard implements Wizard { private final Map<String, CurseStrategy> curses = new HashMap<>(); @Override public String castCurse(String name) { CurseStrategy curse = curses.get(name); if (curse == null) { throw new UnsupportedCurseException(); } return curse.useCurse(); } @Override public void registerCurse(String curseName, CurseStrategy curse) { curses.put(curseName, curse); } } Now, let's update the CurseStrategy interface by adding a method with the @Autowired annotation: Java public interface CurseStrategy { String useCurse(); String curseName(); @Autowired default void registerMe(Wizard wizard) { wizard.registerCurse(curseName(), this); } } At the moment of injecting dependencies, we register our curse into the Wizard. Pros: No loops, and no reliance on inner Spring bean names. Cons: No cons, pure dark magic. Conclusion In this article, we explored the Strategy pattern in the context of Spring. We assessed different strategy injection approaches and demonstrated an optimized solution using Spring's capabilities. The full source code for this article can be found on GitHub. More

Refcard #333

Drupal 9 Essentials

By Cindy McCourt

Exploring Text Generation With Python and GPT-4

By Ashok Gorantla

CORE

Foreign Function and Memory API: Modernizing Native Interfacing in Java 17

By Andrei Tuchin

CORE

Flexible Data Generation With Datafaker Gen

Introduction to Datafaker Datafaker is a modern framework that enables JVM programmers to efficiently generate fake data for their projects using over 200 data providers allowing for quick setup and usage. Custom providers could be written when you need some domain-specific data. In addition to providers, the generated data can be exported to popular formats like CSV, JSON, SQL, XML, and YAML. For a good introduction to the basic features, see "Datafaker: An Alternative to Using Production Data." Datafaker offers many features, such as working with sequences and collections and generating custom objects based on schemas (see "Datafaker 2.0"). Bulk Data Generation In software development and testing, the need to frequently generate data for various purposes arises, whether it's to conduct non-functional tests or to simulate burst loads. Let's consider a straightforward scenario when we have the task of generating 10,000 messages in JSON format to be sent to RabbitMQ. From my perspective, these options are worth considering: Developing your own tool: One option is to write a custom application from scratch to generate these records(messages). If the generated data needs to be more realistic, it makes sense to use Datafaker or JavaFaker. Using specific tools: Alternatively, we could select specific tools designed for particular databases or message brokers. For example, tools like voluble for Kafka provide specialized functionalities for generating and publishing messages to Kafka topics; or a more modern tool like ShadowTraffic, which is currently under development and directed towards a container-based approach, which may not always be necessary. Datafaker Gen: Finally, we have the option to use Datafaker Gen, which I want to consider in the current article. Datafaker Gen Overview Datafaker Gen offers a command-line generator based on the Datafaker library which allows for the continuous generation of data in various formats and integration with different storage systems, message brokers, and backend services. Since this tool uses Datafaker, there may be a possibility that the data is realistic. Configuration of the scheme, format type, and sink can be done without rebuilding the project. Datafake Gen consists of the following main components that can be configured: 1. Schema Definition Users can define the schema for their records in the config.yaml file. The schema specifies the field definitions of the record based on the Datafaker provider. It also allows for the definition of embedded fields. YAML default_locale: en-EN fields: - name: lastname generators: [ Name#lastName ] - name: firstname generators: [ Name#firstName ] 2. Format Datafake Gen allows users to specify the format in which records will be generated. Currently, there are basic implementations for CSV, JSON, SQL, XML, and YAML formats. Additionally, formats can be extended with custom implementations. The configuration for formats is specified in the output.yaml file. YAML formats: csv: quote: "@" separator: $$$$$$$ json: formattedAs: "[]" yaml: xml: pretty: true 3. Sink The sink component determines where the generated data will be stored or published. The basic implementation includes command-line output and text file sinks. Additionally, sinks can be extended with custom implementations such as RabbitMQ, as demonstrated in the current article. The configuration for sinks is specified in the output.yaml file. YAML sinks: rabbitmq: batchsize: 1 # when 1 message contains 1 document, when >1 message contains a batch of documents host: localhost port: 5672 username: guest password: guest exchange: test.direct.exchange routingkey: products.key Extensibility via Java SPI Datafake Gen uses the Java SPI (Service Provider Interface) to make it easy to add new formats or sinks. This extensibility allows for customization of Datafake Gen according to specific requirements. How To Add a New Sink in Datafake Gen Before adding a new sink, you may want to check if it already exists in the datafaker-gen-examples repository. If it does not exist, you can refer to examples on how to add a new sink. When it comes to extending Datafake Gen with new sink implementations, developers have two primary options to consider: By using this parent project, developers can implement sink interfaces for their sink extensions, similar to those available in the datafaker-gen-examples repository. Include dependencies from the Maven repository to access the required interfaces. For this approach, Datafake Gen should be built and exist in the local Maven repository. This approach provides flexibility in project structure and requirements. 1. Implementing RabbitMQ Sink To add a new RabbitMQ sink, one simply needs to implement the net.datafaker.datafaker_gen.sink.Sink interface. This interface contains two methods: getName - This method defines the sink name. run - This method triggers the generation of records and then sends or saves all the generated records to the specified destination. The method parameters include the configuration specific to this sink retrieved from the output.yaml file as well as the data generation function and the desired number of lines to be generated. Java import net.datafaker.datafaker_gen.sink.Sink; public class RabbitMqSink implements Sink { @Override public String getName() { return "rabbitmq"; } @Override public void run(Map<String, ?> config, Function<Integer, ?> function, int numberOfLines) { // Read output configuration ... int numberOfLinesToPrint = numberOfLines; String host = (String) config.get("host"); // Generate lines String lines = (String) function.apply(numberOfLinesToPrint); // Sending or saving results to the expected resource // In this case, this is connecting to RebbitMQ and sending messages. ConnectionFactory factory = getConnectionFactory(host, port, username, password); try (Connection connection = factory.newConnection()) { Channel channel = connection.createChannel(); JsonArray jsonArray = JsonParser.parseString(lines).getAsJsonArray(); jsonArray.forEach(jsonElement -> { try { channel.basicPublish(exchange, routingKey, null, jsonElement.toString().getBytes()); } catch (Exception e) { throw new RuntimeException(e); } }); } catch (Exception e) { throw new RuntimeException(e); } } } 2. Adding Configuration for the New RabbitMQ Sink As previously mentioned, the configuration for sinks or formats can be added to the output.yaml file. The specific fields may vary depending on your custom sink. Below is an example configuration for a RabbitMQ sink: YAML sinks: rabbitmq: batchsize: 1 # when 1 message contains 1 document, when >1 message contains a batch of documents host: localhost port: 5672 username: guest password: guest exchange: test.direct.exchange routingkey: products.key 3. Adding Custom Sink via SPI Adding a custom sink via SPI (Service Provider Interface) involves including the provider configuration in the ./resources/META-INF/services/net.datafaker.datafaker_gen.sink.Sink file. This file contains paths to the sink implementations: Properties files net.datafaker.datafaker_gen.sink.RabbitMqSink These are all 3 simple steps on how to expand Datafake Gen. In this example, we are not providing a complete implementation of the sink, as well as how to use additional libraries. To see the complete implementations, you can refer to the datafaker-gen-rabbitmq module in the example repository. How To Run Step 1 Build a JAR file based on the new implementation: Shell ./mvnw clean verify Step 2 Define the schema for records in the config.yaml file and place this file in the appropriate location where the generator should run. Additionally, define the sinks and formats in the output.yaml file, as demonstrated previously. Step 3 Datafake Gen can be executed through two options: Use bash script from the bin folder in the parent project: Shell # Format json, number of lines 10000 and new RabbitMq Sink bin/datafaker_gen -f json -n 10000 -sink rabbitmq 2. Execute the JAR directly, like this: Shell java -cp [path_to_jar] net.datafaker.datafaker_gen.DatafakerGen -f json -n 10000 -sink rabbitmq How Fast Is It? The test was done based on the scheme described above, which means that one document consists of two fields. Documents are recorded one by one in the RabbitMQ queue in JSON format. The table below shows the speed for 10,000, 100,000, and 1M records on my local machine: Records Time 10000 401 ms 100000 11613ms 1000000 121601ms Conclusion The Datafake Gen tool enables the creation of flexible and fast data generators for various types of destinations. Built on Datafaker, it facilitates realistic data generation. Developers can easily configure the content of records, formats, and sinks to suit their needs. As a simple Java application, it can be deployed anywhere you want, whether it's in Docker or on-premise machines. The full source code is available here. I would like to thank Sergey Nuyanzin for reviewing this article. Thank you for reading, and I am glad to be of help.

By Roman Rybak

JS Toolbox 2024: Runtime Environments and Package Management

Series Introduction Staying ahead of the curve in JavaScript development requires embracing the ever-evolving landscape of tools and technologies. As we navigate through 2024, the landscape of JavaScript development tools will continue to transform, offering more refined, efficient, and user-friendly options. This "JS Toolbox 2024" series is your one-stop shop for a comprehensive overview of the latest and most impactful tools in the JavaScript ecosystem. Across the series, we'll delve into various categories of tools, including runtime environments, package managers, frameworks, static site generators, bundlers, and test frameworks. It will empower you to wield these tools effectively by providing a deep dive into their functionalities, strengths, weaknesses, and how they fit into the modern JavaScript development process. Whether you're a seasoned developer or just starting, this series will equip you with the knowledge needed when it comes to selecting the right tools for your projects in 2024. The series consists of 3 parts: Runtime Environments and Package Management (this article): In this first installment, we explore the intricacies of runtime environments, focusing on Node and Bun. You'll gain insights into their histories, performance metrics, community support, and ease of use, supported by relevant case studies.The segment on package management tools compares npm, yarn, and pnpm, highlighting their performance and security features. We provide tips for choosing the most suitable package manager for your project. Frameworks and Static Site Generators: This post provides a thorough comparison of popular frameworks like React, Vue, Angular, Svelte, and HTMX, focusing on their unique features and suitability for different project types.The exploration of static site generators covers Astro, Nuxt/Next, Hugo, Gatsby, and Jekyll, offering detailed insights into their usability, performance, and community support, along with success stories from real-world applications. Bundlers and Test Frameworks: We delve into the world of bundlers, comparing webpack, build, vite, and parcel 2. This section aims to guide developers through the nuances of each bundler, focusing on their performance, compatibility, and ease of use.The test frameworks section provides an in-depth look at MochaJS, Jest, Jasmine, Puppeteer, Selenium, and Playwright. It includes a comparative analysis emphasizing ease of use, community support, and overall robustness, supplemented with case studies demonstrating their effectiveness in real-world scenarios. Part 1: Runtime Environments and Package Management JavaScript is bigger than ever, and the ecosystem is nothing short of overwhelming. In this JS toolbox 2024 series, we’ve selected and analyzed the most noteworthy JS tools, so that you don’t have to. Just as any durable structure needs a solid foundation, successful JavaScript projects rely heavily on starting with the right tools. This post, the first in our JS Toolbox 2024 series, explores the core pillars of the JavaScript and TypeScript ecosystem: Runtime environments, package management, and development servers. In this post: Runtime environments Node.js Deno Bun 2. Comparing JS runtimes Installation Performance, stability, and security Community 3. Package managers NPM Yarn pnpM Bun 4. What to choose Runtime Environments In JavaScript development, runtimes are the engines that drive advanced, server-centric projects beyond the limitations of a user's browser. This independence is pivotal in modern web development, allowing for more sophisticated and versatile applications. The JavaScript runtime market is more dynamic than ever, with several contenders competing for the top spot. Node.js, the long-established leader in this space, now faces formidable competition from Deno and Bun. Deno is the brainchild of Ryan Dahl, the original creator of Node.js. It represents a significant step forward in runtime technology, emphasizing security through fine-grained access controls and modern capabilities like native TypeScript support. Bun has burst onto the scene, releasing version 1.0 in September 2023. Bun sets itself apart with exceptional speed, challenging the performance standards established by its predecessors. Bun's rapid execution capabilities, enabled by just-in-time (JIT) execution, make it a powerful alternative in the runtime environment space. An overview of framework popularity trends The popularity of Node.js has continued to grow over 2023, and I anticipate this will continue into 2024. There has been a slight downtrend in the growth trajectory, which I’d guess is due to the other tooling growing in market share. Deno has seen substantial growth over 2023. If the current trend continues I anticipate Deno to overtake Node.js in popularity in 2024, though it’s worth mentioning that star-based popularity doesn’t reflect usage in the field. Without a doubt, Node.js will retain its position as the lead environment for production systems throughout 2024. Bun has seen the largest growth in this category over the past year. I anticipate that Bun will find a steady foothold and continue its ascent, following the release of version 1.0. It’s early days for this new player, but comparing early-stage growth to others in the category, it’s shaping up to be a high performer. Node.js Node.js, acclaimed as the leading web technology by StackOverflow developers, has been a significant player in the web development world since its inception in 2009. It revolutionized web development by enabling JavaScript for server-side scripting, thus allowing for the creation of complex, backend-driven applications. Advantages Asynchronous and event-driven: Node.js operates on an asynchronous, event-driven architecture, making it efficient for scalable network applications. This model allows Node.js to handle multiple operations concurrently without blocking the main thread. Rich ecosystem: With a diverse and extensive range of tools, resources, and libraries available, Node.js offers developers an incredibly rich ecosystem, supporting a wide array of development needs. Optimized for performance: Node.js is known for its low-latency handling of HTTP requests, which is optimal for web frameworks. It efficiently utilizes system resources, allowing for load balancing and the use of multiple cores through child processes and its cluster module. Disadvantages Learning curve for asynchronous programming: The non-blocking, asynchronous nature of Node.js can be challenging for developers accustomed to linear programming paradigms, leading to a steep learning curve. Callback hell: While manageable, Node.js can lead to complex nested callbacks – often referred to as "callback hell" – which can make code difficult to read and maintain. However, this can be mitigated with modern features like async/await. Deno Deno represents a step forward in JavaScript and TypeScript runtimes, leveraging Google’s V8 engine and built-in Rust for enhanced security and performance. Conceived by Ryan Dahl, the original creator of Node.js, Deno is positioned as a more secure and modern alternative, addressing some of the core issues found in Node.js, particularly around security. Advantages Enhanced security: Deno's secure-by-default approach requires explicit permissions for file, network, and environment access, reducing the risks associated with an all-access runtime. Native TypeScript support: It offers first-class support for TypeScript and TSX, allowing developers to use TypeScript out of the box without additional transpiling steps. Single executable compilation: Deno can compile entire applications into a single, self-contained executable, simplifying deployment and distribution processes. Disadvantages Young ecosystem: Being relatively new compared to Node.js, Deno’s ecosystem is still growing, which may temporarily limit the availability of third-party modules and tools. Adoption barrier: For teams and projects deeply integrated with Node.js, transitioning to Deno can represent a significant change, posing challenges in terms of adoption and migration. Bun Bun emerges as a promising new contender in the JavaScript runtime space, positioning itself as a faster and more efficient alternative to Node.js. Developed using Zig and powered by JavaScriptCore, Bun is designed to deliver significantly quicker startup times and lower memory usage, making it an attractive option for modern web development. Currently, Bun provides a limited, experimental native build for Windows with full support for Linux and macOS. Hopefully, early in 2024, we see full support for Windows released. Advantages High performance: Bun's main draw is its performance, offering faster execution and lower resource usage compared to traditional runtimes, making it particularly suitable for high-efficiency requirements. Integrated development tools: It comes with an integrated suite of tools, including a test runner, script runner, and a Node.js-compatible package manager, all optimized for speed and compatibility with Node.js projects. Evolving ecosystem: Bun is continuously evolving, with a focus on enhancing Node.js compatibility and broadening its integration with various frameworks, signaling its potential as a versatile and adaptable solution for diverse development needs. Disadvantages Relative newness in the market: As a newer player, Bun's ecosystem is not as mature as Node.js, which might pose limitations in terms of available libraries and community support. Compatibility challenges: While efforts are being made to improve compatibility with Node.js, there may still be challenges and growing pains in integrating Bun into existing Node.js-based projects or workflows. Comparing JavaScript Runtimes Installation Each JavaScript runtime has its unique installation process. Here's a brief overview of how to install Node.js, Deno, and Bun: Node.js Download: Visit the Node.js website and download the installer suitable for your operating system. Run installer: Execute the downloaded file and follow the installation prompts. This process will install both Node.js and npm. Verify installation: Open a terminal or command prompt and type node -v and npm -v to check the installed versions of Node.js and npm, respectively. Managing different versions of Node.js has historically been a challenge for developers. To address this issue, tools like NVM (Node Version Manager) and NVM Windows have been developed, greatly simplifying the process of installing and switching between various Node.js versions. Deno Shell Command: You can install Deno using a simple shell command.• Windows: irm https://deno.land/install.ps1 | iex• Linux/macOS: curl -fsSL https://deno.land/x/install/install.sh | sh Alternative methods: Other methods like downloading a binary from the Deno releases page are also available. Verify installation: To ensure Deno is installed correctly, type deno --version in your terminal. Bun Shell Command: Similar to Deno, Bun can be installed using a shell command. For instance, on macOS, Linux, and WSL use the command curl https://bun.sh/install | bash. Alternative methods: For detailed instructions or alternative methods, check the Bun installation guide. Verify installation: After installation, run bun --version in your terminal to verify that Bun is correctly installed. Performance, Stability, and Security In evaluating JavaScript runtimes, performance, stability, and security are the key factors to consider. Mayank Choubey's benchmark studies provide insightful comparisons among Node.js, Deno, and Bun: Node.js vs Deno vs Bun: Express hello world server benchmarking Node.js vs Deno vs Bun: Native HTTP hello world server benchmarking I’d recommend giving the post a read if you’re interested in the specifics. Otherwise, I’ll do my best to summarize the results below. Node.js Historically, Node.js has been known for its efficient handling of asynchronous operations and has set a standard in server-side JavaScript performance. In the benchmark, Node.js displayed solid performance, reflective of its maturity and optimization over the years. However, it didn't lead the pack in terms of raw speed. As Node.js has been around for a long time and has proven its reliability, it wins the category of stability. Deno Deno, being a relatively newer runtime, has shown promising improvements in performance, particularly in the context of security and TypeScript support. The benchmark results for Deno were competitive, showcasing its capability to handle server requests efficiently, though it still trails slightly behind in raw processing speed compared to Bun. Given its emphasis on security features like explicit permissions for file, network, and environment access, Deno excels in the category of security. Bun Bun made a significant impression with its performance in this benchmark. It leverages Zig and JavaScriptCore, which contributes to its faster startup times and lower memory usage. In the "Hello World" server test, Bun outperformed both Node.js and Deno in terms of request handling speed, showcasing its potential as a high-performance JavaScript runtime. With its significant speed improvements, Bun leads in the category of performance. These results suggest that while Node.js remains a reliable and robust choice for many applications, Deno and Bun are catching up, offering competitive and sometimes superior performance metrics. Bun, in particular, demonstrates remarkable speed, which could be a game-changer for performance-critical applications. However, it's important to consider other factors such as stability, community support, and feature completeness when choosing a runtime for your project. Community The community surrounding a JavaScript runtime is vital for its growth and evolution. It shapes development, provides support, and drives innovation. Let's briefly examine the community dynamics for Node.js, Deno, and Bun: Node.js: Node.js has one of the largest, most diverse communities in software development, enriched by a wide array of libraries, tools, and resources. Its community actively contributes to its core and modules, bolstered by global events and forums for learning and networking. Deno: Deno's community is rapidly growing, drawing developers with its modern and security-centric features. It's characterized by active involvement in the runtime’s development and a strong online presence, particularly on platforms like GitHub and Discord. Bun: Although newer, Bun’s community is dynamic and quickly expanding. Early adopters are actively engaged in its development and performance enhancement, with lively discussions and feedback exchanges on online platforms. Each of these communities, from Node.js’s well-established network to the emerging groups around Deno and Bun, plays a crucial role in the adoption and development of these runtimes. For developers, understanding the nuances of these communities can be key to leveraging the full potential of a chosen runtime. Package Managers If you’ve ever worked on the front end of a modern web application or if you're a full-stack node engineer, you’ve likely used a package manager at some point. The package manager is responsible for managing the dependencies of your project, such as libraries, frameworks, and utilities. NPM is the default package manager that comes pre-installed with Node.js. Yarn and PNPM compete to take NPM's spot as the package management tool of choice for developers working in the JavaScript ecosystem. An overview of framework popularity trends NPM Node Package Manager or NPM for short, is the default and most dominant package manager for JavaScript projects. It comes pre-installed with Node.js, providing developers with immediate access to the npm registry, allowing them to install, share, and manage package dependencies right from the start of their project. It was created in 2009 by Isaac Schlueter as a way to share and reuse code for Node.js projects. Since then, it has grown to become a huge repository of packages that can be used for both front-end and back-end development. NPM consists of two main components: NPM CLI (Command Line Interface): This tool is used by developers to install, update, and manage packages (libraries or modules) in their JavaScript projects. It interacts with npm’s online repository, allowing developers to add external packages to their projects easily. NPM registry: An extensive online database of public and private JavaScript packages, the npm Registry is where developers can publish their packages, making them accessible to the wider JavaScript community. It's known for its vast collection of libraries, frameworks, and tools, contributing to the versatility and functionality of JavaScript projects. This star graph doesn’t capture much in terms of the overall popularity of NPM CLI given that this tool comes pre-installed with Node.js. Knowing this, it’s worth also reviewing the overall download count of these packages. NPM currently has 56,205,118,637 weekly downloads Woah, 56.2B! It’s safe to say NPM isn’t going anywhere. From the graphs, we can see a steady incline in the overall popularity of this tool through 2023. I predict this growth will continue through 2024. Yarn Yarn is a well-established open-source package manager created in 2016 by Facebook, Google, Exponent, and Tilde. It was designed to address some of the issues and limitations of NPM, such as speed, correctness, security, and developer experience. To improve these areas, Yarn incorporates a range of innovative features. These include workspaces for managing multiple packages within a single repository, offline caching for faster installs, parallel installations for improved speed, a hardened mode for enhanced security, and interactive commands for a more intuitive user interface. These features collectively contribute to Yarn’s robustness and efficiency. It features a command-line interface that closely resembles NPM's but with several enhancements and differences. It utilizes the same package.json file as NPM for defining project dependencies. Additionally, Yarn introduces the yarn.lock file, which precisely locks down the versions of dependencies, ensuring consistent installs across environments. Like NPM, Yarn also creates a node_modules folder where it installs and organizes the packages for your project. Yarn currently has 4,396,069 weekly downloads Given that Yarn and pnpM require manual installs this does mean the download counts are un-comparable with NPM but it still gives us a glance at the overall trends. In 2023, Yarn appears to have lost some of its growth trajectory but still remains the most popular alternative to NPM for package management. pnpM Performant NPM or pnpM for short, is another alternative package manager for JavaScript that was created in 2016 by Zoltan Kochan. It was designed to be faster, lighter, and more secure than both NPM and Yarn. It excels in saving disk space and speeding up the installation process. Unlike npm, where each project stores separate copies of dependencies, pnpm stores them in a content-addressable store. This approach means if multiple projects use the same dependency, they share a single stored copy, significantly reducing disk usage. When updating dependencies, pnpm only adds changed files instead of duplicating the entire package. The installation process in pnpM is streamlined into three stages: resolving dependencies, calculating the directory structure, and linking dependencies, making it faster than traditional methods. pnpM also creates a unique node_modules directory using symlinks for direct dependencies only, avoiding unnecessary access to indirect dependencies. This approach ensures a cleaner dependency structure, while still offering a traditional flat structure option through its node-linker setting for those who prefer it. pnpM currently has 8,016,757 weekly downloads pnpM's popularity surged in 2023, and I foresee this upward trend extending into 2024, as an increasing number of developers recognize its resource efficiency and streamlined project setup. Bun As Bun comes with an npm-compatible package manager, I felt it was worth mentioning here. I've covered Bun in the "Runtime Environments" section above. What To Choose Choosing the right tool for your project in 2024 depends on a variety of factors including your project's specific requirements, your team's familiarity with the technology, and the particular strengths of each tool. In the dynamic world of JavaScript development, having a clear understanding of these factors is crucial for making an informed decision. For those prioritizing stability and a proven track record, Node.js remains a top recommendation. It's well-established, supported by a vast ecosystem, and continues to be a reliable choice for a wide range of applications. Node.js's maturity makes it a safe bet, especially for projects where long-term viability and extensive community support are essential. On the other hand, if you're inclined towards experimenting with the latest advancements in the field and are operating in a Linux-based environment, Bun presents an exciting opportunity. It stands out for its impressive performance and is ideal for those looking to leverage the bleeding edge of JavaScript runtime technology. Bun’s rapid execution capabilities make it a compelling option for performance-driven projects. When it comes to package management, pnpM is an excellent choice. Its efficient handling of dependencies and disk space makes it ideal for developers managing multiple projects or large dependencies. With its growing popularity and focus on performance, pnpM is well-suited for modern JavaScript development. JavaScript tools in 2024 offer a massive range catered to different needs and preferences. Whether you opt for the stability of Node.js, the cutting-edge performance of Bun, or the efficient dependency management of pnpM, each tool brings unique strengths to the table. Carefully consider your project’s requirements and team’s expertise to make the best choice for your development journey in 2024. Like you, I’m always curious and looking to learn. If I've overlooked a noteworthy tool or if you have any feedback to share, reach out on LinkedIn.

By Ollie Bannister

CRUDing NoSQL Data With Quarkus, Part One: MongoDB

MongoDB is one of the most reliable and robust document-oriented NoSQL databases. It allows developers to provide feature-rich applications and services with various modern built-in functionalities, like machine learning, streaming, full-text search, etc. While not a classical relational database, MongoDB is nevertheless used by a wide range of different business sectors and its use cases cover all kinds of architecture scenarios and data types. Document-oriented databases are inherently different from traditional relational ones where data are stored in tables and a single entity might be spread across several such tables. In contrast, document databases store data in separate unrelated collections, which eliminates the intrinsic heaviness of the relational model. However, given that the real world's domain models are never so simplistic to consist of unrelated separate entities, document databases (including MongoDB) provide several ways to define multi-collection connections similar to the classical databases relationships, but much lighter, more economical, and more efficient. Quarkus, the "supersonic and subatomic" Java stack, is the new kid on the block that the most trendy and influential developers are desperately grabbing and fighting over. Its modern cloud-native facilities, as well as its contrivance (compliant with the best-of-the-breed standard libraries), together with its ability to build native executables have seduced Java developers, architects, engineers, and software designers for a couple of years. We cannot go into further details here of either MongoDB or Quarkus: the reader interested in learning more is invited to check the documentation on the official MongoDB website or Quarkus website. What we are trying to achieve here is to implement a relatively complex use case consisting of CRUDing a customer-order-product domain model using Quarkus and its MongoDB extension. In an attempt to provide a real-world inspired solution, we're trying to avoid simplistic and caricatural examples based on a zero-connections single-entity model (there are dozens nowadays). So, here we go! The Domain Model The diagram below shows our customer-order-product domain model: As you can see, the central document of the model is Order, stored in a dedicated collection named Orders. An Order is an aggregate of OrderItem documents, each of which points to its associated Product. An Order document also references the Customer who placed it. In Java, this is implemented as follows: Java @MongoEntity(database = "mdb", collection="Customers") public class Customer { @BsonId private Long id; private String firstName, lastName; private InternetAddress email; private Set<Address> addresses; ... } The code above shows a fragment of the Customer class. This is a POJO (Plain Old Java Object) annotated with the @MongoEntity annotation, which parameters define the database name and the collection name. The @BsonId annotation is used in order to configure the document's unique identifier. While the most common use case is to implement the document's identifier as an instance of the ObjectID class, this would introduce a useless tidal coupling between the MongoDB-specific classes and our document. The other properties are the customer's first and last name, the email address, and a set of postal addresses. Let's look now at the Order document. Java @MongoEntity(database = "mdb", collection="Orders") public class Order { @BsonId private Long id; private DBRef customer; private Address shippingAddress; private Address billingAddress; private Set<DBRef> orderItemSet = new HashSet<>() ... } Here we need to create an association between an order and the customer who placed it. We could have embedded the associated Customer document in our Order document, but this would have been a poor design because it would have redundantly defined the same object twice. We need to use a reference to the associated Customer document, and we do this using the DBRef class. The same thing happens for the set of the associated order items where, instead of embedding the documents, we use a set of references. The rest of our domain model is quite similar and based on the same normalization ideas; for example, the OrderItem document: Java @MongoEntity(database = "mdb", collection="OrderItems") public class OrderItem { @BsonId private Long id; private DBRef product; private BigDecimal price; private int amount; ... } We need to associate the product which makes the object of the current order item. Last but not least, we have the Product document: Java @MongoEntity(database = "mdb", collection="Products") public class Product { @BsonId private Long id; private String name, description; private BigDecimal price; private Map<String, String> attributes = new HashMap<>(); ... } That's pretty much all as far as our domain model is concerned. There are, however, some additional packages that we need to look at: serializers and codecs. In order to be able to be exchanged on the wire, all our objects, be they business or purely technical ones, have to be serialized and deserialized. These operations are the responsibility of specially designated components called serializers/deserializers. As we have seen, we're using the DBRef type in order to define the association between different collections. Like any other object, a DBRef instance should be able to be serialized/deserialized. The MongoDB driver provides serializers/deserializers for the majority of the data types supposed to be used in the most common cases. However, for some reason, it doesn't provide serializers/deserializers for the DBRef type. Hence, we need to implement our own one, and this is what the serializers package does. Let's look at these classes: Java public class DBRefSerializer extends StdSerializer<DBRef> { public DBRefSerializer() { this(null); } protected DBRefSerializer(Class<DBRef> dbrefClass) { super(dbrefClass); } @Override public void serialize(DBRef dbRef, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException { if (dbRef != null) { jsonGenerator.writeStartObject(); jsonGenerator.writeStringField("id", (String)dbRef.getId()); jsonGenerator.writeStringField("collectionName", dbRef.getCollectionName()); jsonGenerator.writeStringField("databaseName", dbRef.getDatabaseName()); jsonGenerator.writeEndObject(); } } } This is our DBRef serializer and, as you can see, it's a Jackson serializer. This is because the quarkus-mongodb-panache extension that we're using here relies on Jackson. Perhaps, in a future release, JSON-B will be used but, for now, we're stuck with Jackson. It extends the StdSerializer class as usual and serializes its associated DBRef object by using the JSON generator, passed as an input argument, to write on the output stream the DBRef components; i.e., the object ID, the collection name, and the database name. For more information concerning the DBRef structure, please see the MongoDB documentation. The deserializer is performing the complementary operation, as shown below: Java public class DBRefDeserializer extends StdDeserializer<DBRef> { public DBRefDeserializer() { this(null); } public DBRefDeserializer(Class<DBRef> dbrefClass) { super(dbrefClass); } @Override public DBRef deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException, JacksonException { JsonNode node = jsonParser.getCodec().readTree(jsonParser); return new DBRef(node.findValue("databaseName").asText(), node.findValue("collectionName").asText(), node.findValue("id").asText()); } } This is pretty much all that may be said as far as the serializers/deserializers are concerned. Let's move further to see what the codecs package brings to us. Java objects are stored in a MongoDB database using the BSON (Binary JSON) format. In order to store information, the MongoDB driver needs the ability to map Java objects to their associated BSON representation. It does that on behalf of the Codec interface, which contains the required abstract methods for the mapping of the Java objects to BSON and the other way around. Implementing this interface, one can define the conversion logic between Java and BSON, and conversely. The MongoDB driver includes the required Codec implementation for the most common types but again, for some reason, when it comes to DBRef, this implementation is only a dummy one, which raises UnsupportedOperationException. Having contacted the MongoDB driver implementers, I didn't succeed in finding any other solution than implementing my own Codec mapper, as shown by the class DocstoreDBRefCodec. For brevity reasons, we won't reproduce this class' source code here. Once our dedicated Codec is implemented, we need to register it with the MongoDB driver, such that it uses it when it comes to mapping DBRef types to Java objects and conversely. In order to do that, we need to implement the interface CoderProvider which, as shown by the class DocstoreDBRefCodecProvider, returns via its abstract get() method, the concrete class responsible for performing the mapping; i.e., in our case, DocstoreDBRefCodec. And that's all we need to do here as Quarkus will automatically discover and use our CodecProvider customized implementation. Please have a look at these classes to see and understand how things are done. The Data Repositories Quarkus Panache greatly simplifies the data persistence process by supporting both the active record and the repository design patterns. Here, we'll be using the second one. As opposed to similar persistence stacks, Panache relies on the compile-time bytecode enhancements of the entities. It includes an annotation processor that automatically performs these enhancements. All that this annotation processor needs in order to perform its enhancements job is an interface like the one below: Java @ApplicationScoped public class CustomerRepository implements PanacheMongoRepositoryBase<Customer, Long>{} The code above is all that you need in order to define a complete service able to persist Customer document instances. Your interface needs to extend the PanacheMongoRepositoryBase one and parameter it with your object ID type, in our case a Long. The Panache annotation processor will generate all the required endpoints required to perform the most common CRUD operations, including but not limited to saving, updating, deleting, querying, paging, sorting, transaction handling, etc. All these details are fully explained here. Another possibility is to extend PanacheMongoRepository instead of PanacheMongoRepositoryBase and to use the provided ObjectID keys instead of customizing them as Long, as we did in our example. Whether you chose the 1st or the 2nd alternative, this is only a preference matter. The REST API In order for our Panache-generated persistence service to become effective, we need to expose it through a REST API. In the most common case, we have to manually craft this API, together with its implementation, consisting of the full set of the required REST endpoints. This fastidious and repetitive operation might be avoided by using the quarkus-mongodb-rest-data-panache extension, which annotation processor is able to automatically generate the required REST endpoints, out of interfaces having the following pattern: Java public interface CustomerResource extends PanacheMongoRepositoryResource<CustomerRepository, Customer, Long> {} Believe it if you want: this is all you need to generate a full REST API implementation with all the endpoints required to invoke the persistence service generated previously by the mongodb-panache extension annotation processor. Now we are ready to build our REST API as a Quarkus microservice. We chose to build this microservice as a Docker image, on behalf of the quarkus-container-image-jib extension. By simply including the following Maven dependency: XML <dependency> <groupId>io.quarkus</groupId> <artifactId>quarkus-container-image-jib</artifactId> </dependency> The quarkus-maven-plugin will create a local Docker image to run our microservice. The parameters of this Docker image are defined by the application.properties file, as follows: Properties files quarkus.container-image.build=true quarkus.container-image.group=quarkus-nosql-tests quarkus.container-image.name=docstore-mongodb quarkus.mongodb.connection-string = mongodb://admin:admin@mongo:27017 quarkus.mongodb.database = mdb quarkus.swagger-ui.always-include=true quarkus.jib.jvm-entrypoint=/opt/jboss/container/java/run/run-java.sh Here we define the name of the newly created Docker image as being quarkus-nosql-tests/docstore-mongodb. This is the concatenation of the parameters quarkus.container-image.group and quarkus.container-image.name separated by a "/". The property quarkus.container-image.build having the value true instructs the Quarkus plugin to bind the build operation to the package phase of maven. This way, simply executing a mvn package command, we generate a Docker image able to run our microservice. This may be tested by running the docker images command. The property named quarkus.jib.jvm-entrypoint defines the command to be run by the newly generated Docker image. quarkus-run.jar is the Quarkus microservice standard startup file used when the base image is ubi8/openjdk-17-runtime, as in our case. Other properties are quarkus.mongodb.connection-string and quarkus.mongodb.database = mdb which define the MongoDB database connection string and the name of the database. Last but not least, the property quarkus.swagger-ui.always-include includes the Swagger UI interface in our microservice space such that it allows us to test it easily. Let's see now how to run and test the whole thing. Running and Testing Our Microservices Now that we looked at the details of our implementation, let's see how to run and test it. We chose to do it on behalf of the docker-compose utility. Here is the associated docker-compose.yml file: YAML version: "3.7" services: mongo: image: mongo environment: MONGO_INITDB_ROOT_USERNAME: admin MONGO_INITDB_ROOT_PASSWORD: admin MONGO_INITDB_DATABASE: mdb hostname: mongo container_name: mongo ports: - "27017:27017" volumes: - ./mongo-init/:/docker-entrypoint-initdb.d/:ro mongo-express: image: mongo-express depends_on: - mongo hostname: mongo-express container_name: mongo-express links: - mongo:mongo ports: - 8081:8081 environment: ME_CONFIG_MONGODB_ADMINUSERNAME: admin ME_CONFIG_MONGODB_ADMINPASSWORD: admin ME_CONFIG_MONGODB_URL: mongodb://admin:admin@mongo:27017/ docstore: image: quarkus-nosql-tests/docstore-mongodb:1.0-SNAPSHOT depends_on: - mongo - mongo-express hostname: docstore container_name: docstore links: - mongo:mongo - mongo-express:mongo-express ports: - "8080:8080" - "5005:5005" environment: JAVA_DEBUG: "true" JAVA_APP_DIR: /home/jboss JAVA_APP_JAR: quarkus-run.jar This file instructs the docker-compose utility to run three services: A service named mongo running the Mongo DB 7 database A service named mongo-express running the MongoDB administrative UI A service named docstore running our Quarkus microservice We should note that the mongo service uses an initialization script mounted on the docker-entrypoint-initdb.d directory of the container. This initialization script creates the MongoDB database named mdb such that it could be used by the microservices. JavaScript db = db.getSiblingDB(process.env.MONGO_INITDB_ROOT_USERNAME); db.auth( process.env.MONGO_INITDB_ROOT_USERNAME, process.env.MONGO_INITDB_ROOT_PASSWORD, ); db = db.getSiblingDB(process.env.MONGO_INITDB_DATABASE); db.createUser( { user: "nicolas", pwd: "password1", roles: [ { role: "dbOwner", db: "mdb" }] }); db.createCollection("Customers"); db.createCollection("Products"); db.createCollection("Orders"); db.createCollection("OrderItems"); This is an initialization JavaScript that creates a user named nicolas and a new database named mdb. The user has administrative privileges on the database. Four new collections, respectively named Customers, Products, Orders and OrderItems, are created as well. In order to test the microservices, proceed as follows: Clone the associated GitHub repository: $ git clone https://github.com/nicolasduminil/docstore.git Go to the project: $ cd docstore Build the project: $ mvn clean install Check that all the required Docker containers are running: $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7882102d404d quarkus-nosql-tests/docstore-mongodb:1.0-SNAPSHOT "/opt/jboss/containe…" 8 seconds ago Up 6 seconds 0.0.0.0:5005->5005/tcp, :::5005->5005/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 8443/tcp docstore 786fa4fd39d6 mongo-express "/sbin/tini -- /dock…" 8 seconds ago Up 7 seconds 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp mongo-express 2e850e3233dd mongo "docker-entrypoint.s…" 9 seconds ago Up 7 seconds 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp mongo Run the integration tests: $ mvn -DskipTests=false failsafe:integration-test This last command will run all the integration tests which should succeed. These integration tests are implemented using the RESTassured library. The listing below shows one of these integration tests located in the docstore-domain project: Java @QuarkusIntegrationTest @TestMethodOrder(MethodOrderer.OrderAnnotation.class) public class CustomerResourceIT { private static Customer customer; @BeforeAll public static void beforeAll() throws AddressException { customer = new Customer("John", "Doe", new InternetAddress("john.doe@gmail.com")); customer.addAddress(new Address("Gebhard-Gerber-Allee 8", "Kornwestheim", "Germany")); customer.setId(10L); } @Test @Order(10) public void testCreateCustomerShouldSucceed() { given() .header("Content-type", "application/json") .and().body(customer) .when().post("/customer") .then() .statusCode(HttpStatus.SC_CREATED); } @Test @Order(20) public void testGetCustomerShouldSucceed() { assertThat (given() .header("Content-type", "application/json") .when().get("/customer") .then() .statusCode(HttpStatus.SC_OK) .extract().body().jsonPath().getString("firstName[0]")).isEqualTo("John"); } @Test @Order(30) public void testUpdateCustomerShouldSucceed() { customer.setFirstName("Jane"); given() .header("Content-type", "application/json") .and().body(customer) .when().pathParam("id", customer.getId()).put("/customer/{id}") .then() .statusCode(HttpStatus.SC_NO_CONTENT); } @Test @Order(40) public void testGetSingleCustomerShouldSucceed() { assertThat (given() .header("Content-type", "application/json") .when().pathParam("id", customer.getId()).get("/customer/{id}") .then() .statusCode(HttpStatus.SC_OK) .extract().body().jsonPath().getString("firstName")).isEqualTo("Jane"); } @Test @Order(50) public void testDeleteCustomerShouldSucceed() { given() .header("Content-type", "application/json") .when().pathParam("id", customer.getId()).delete("/customer/{id}") .then() .statusCode(HttpStatus.SC_NO_CONTENT); } @Test @Order(60) public void testGetSingleCustomerShouldFail() { given() .header("Content-type", "application/json") .when().pathParam("id", customer.getId()).get("/customer/{id}") .then() .statusCode(HttpStatus.SC_NOT_FOUND); } } You can also use the Swagger UI interface for testing purposes by firing your preferred browser at http://localhost:8080/q:swagger-ui. Then, in order to test endpoints, you can use the payloads in the JSON files located in the src/resources/data directory of the docstore-api project. You also can use the MongoDB UI administrative interface by going to http://localhost:8081 and authenticating yourself with the default credentials (admin/pass). You can find the project source code in my GitHub repository. Enjoy!

By Nicolas Duminil

CORE

Scaling Java Microservices to Extreme Performance Using NCache

Microservices have emerged as a transformative architectural approach in the realm of software development, offering a paradigm shift from monolithic structures to a more modular and scalable system. At its core, microservices involve breaking down complex applications into smaller, independently deployable services that communicate seamlessly, fostering agility, flexibility, and ease of maintenance. This decentralized approach allows developers to focus on specific functionalities, enabling rapid development, continuous integration, and efficient scaling to meet the demands of modern, dynamic business environments. As organizations increasingly embrace the benefits of microservices, this article explores the key principles, advantages, and challenges associated with this architectural style, shedding light on its pivotal role in shaping the future of software design and deployment. A fundamental characteristic of microservices applications is the ability to design, develop, and deploy each microservice independently, utilizing diverse technology stacks. Each microservice functions as a self-contained, autonomous application with its own dedicated persistent storage, whether it be a relational database, a NoSQL DB, or even a legacy file storage system. This autonomy enables individual microservices to scale independently, facilitating seamless real-time infrastructure adjustments and enhancing overall manageability. NCache Caching Layer in Microservice Architecture In scenarios where application transactions surge, bottlenecks may persist, especially in architectures where microservices store data in non-scalable relational databases. Simply deploying additional instances of the microservice doesn't alleviate the problem. To address these challenges, consider integrating NCache as a distributed cache at the caching layer between microservices and datastores. NCache serves not only as a cache but also functions as a scalable in-memory publisher/subscriber messaging broker, facilitating asynchronous communication between microservices. Microservice Java application performance optimization can be achieved by the cache techniques like Cache item locking, grouping Cache data, Hibernate Caching, SQL Query, data structure, spring data cache technique pub-sub messaging, and many more with NCache. Please check the out-of-the-box features provided by NCache. Using NCache as Hibernate Second Level Java Cache Hibernate First-Level Cache The Hibernate first-level cache serves as a fundamental standalone (in-proc) cache linked to the Session object, limited to the current session. Nonetheless, a drawback of the first-level cache is its inability to share objects between different sessions. If the same object is required by multiple sessions, each triggers a database trip to load it, intensifying database traffic and exacerbating scalability issues. Furthermore, when the session concludes, all cached data is lost, necessitating a fresh fetch from the database upon the next retrieval. Hibernate Second-Level Cache For high-traffic Hibernate applications relying solely on the first-level cache, deployment in a web farm introduces challenges related to cache synchronization across servers. In a web farm setup, each node operates a web server—such as Apache, Oracle WebLogic, etc.—with multiple instances of httpd processes to serve requests. Each Hibernate first-level cache in these HTTP worker processes maintains a distinct version of the same data directly cached from the database, posing synchronization issues. This is why Hibernate offers a second-level cache with a provider model. The Hibernate second-level cache enables you to integrate third-party distributed (out-proc) caching providers to cache objects across sessions and servers. Unlike the first-level cache, the second-level cache is associated with the SessionFactory object and is accessible to the entire application, extending beyond a single session. Enabling the Hibernate second-level cache results in the coexistence of two caches: the first-level cache and the second-level cache. Hibernate endeavors to retrieve objects from the first-level cache first; if unsuccessful, it attempts to fetch them from the second-level cache. If both attempts fail, the objects are directly loaded from the database and cached. This configuration substantially reduces database traffic, as a significant portion of the data is served by the second-level distributed cache. NCache Java has implemented a Hibernate second-level caching provider by extending org.hibernate.cache.CacheProvider. Integrating NCache Java Hibernate distributed caching provider with the Hibernate application requires no code changes. This integration enables you to scale your Hibernate application to multi-server configurations without the database becoming a bottleneck. NCache also delivers enterprise-level distributed caching features, including data size management, data synchronization across servers, and more. To incorporate the NCache Java Hibernate caching provider, a simple modification of your hibernate.cfg.xml and ncache.xml is all that is required. Thus, with the NCache Java Hibernate distributed cache provider, you can achieve linear scalability for your Hibernate applications seamlessly, requiring no alterations to your existing code. Code Snippet Java // Configure Hibernate properties programmatically Properties hibernateProperties = new Properties(); hibernateProperties.put("hibernate.connection.driver_class", "org.h2.Driver"); hibernateProperties.put("hibernate.connection.url", "jdbc:h2:mem:testdb"); hibernateProperties.put("hibernate.show_sql", "false"); hibernateProperties.put("hibernate.hbm2ddl.auto", "create-drop"); hibernateProperties.put("hibernate.cache.use_query_cache", "true"); hibernateProperties.put("hibernate.cache.use_second_level_cache", "true"); hibernateProperties.put("hibernate.cache.region.factory_class", "org.hibernate.cache.jcache.internal.JCacheRegionFactory"); hibernateProperties.put("hibernate.javax.cache.provider", "com.alachisoft.ncache.hibernate.jcache.HibernateNCacheCachingProvider"); // Set other Hibernate properties as needed Configuration configuration = new Configuration() .setProperties(hibernateProperties).addAnnotatedClass(Product.class); Logger.getLogger("org.hibernate").setLevel(Level.OFF); // Build the ServiceRegistry ServiceRegistry serviceRegistry = new StandardServiceRegistryBuilder() .applySettings(configuration.getProperties()).build(); // Build the SessionFactory SessionFactory factory = configuration.buildSessionFactory(serviceRegistry); // Create a List of Product objects ArrayList<Product> products = (ArrayList<Product>) getProducts(); // Open a new Hibernate session to save products to the database. This also caches it try (Session session = factory.openSession()) { Transaction transaction = session.beginTransaction(); // save() method saves products to the database and caches it too System.out.println("ProductID, Name, Price, Category"); for (Product product : products) { System.out.println("- " + product.getProductID() + ", " + product.getName() + ", " + product.getPrice() + ", " + product.getCategory()); session.save(product); } transaction.commit(); System.out.println(); // Now open a new session to fetch products from the DB. // But, these products are actually fetched from the cache try (Session session = factory.openSession()) { List<Product> productList = (List<Product>) session.createQuery("from Product").list(); if (productList != null) { printProductDetails(productList); } } Integrate NCache with Hibernate to effortlessly cache the results of queries. When these objects are subsequently fetched by Hibernate, they are retrieved from the cache, thereby avoiding a costly trip to the database. From the above code sample, the products are saved in the database, and it also caches; now, when the new session opens to fetch the product details, it will fetch from the Cache and avoid an unnecessary database trip. Learn more about Hibernate Caching Scaling With NCache Pub/Sub Messaging NCache is a distributed in-memory caching solution designed for .NET. Its compatibility extends to Java through a native client and third-party integrations, ensuring seamless support for both platforms. NCache serves as an in-memory distributed data store tailored for .NET and Java, offering a feature-rich, in-memory pub/sub mechanism for event-driven communication. This makes it straightforward to set up NCache as a messaging broker, employing the Pub/Sub model for seamless asynchronous communication between microservices. Using NCache In-Memory Pub/Sub for Microservices NCache enables Pub/Sub functionality by establishing a topic where microservices can publish and subscribe to events. These events are published to the NCache message broker outside the microservice. Within each subscribing microservice, there exists an event handler to manage the corresponding event once it has been published by the originating microservice. In the realm of Java microservices, NCache functions as an event bus or message broker, facilitating the relay of messages to one or multiple subscribers. In the context of Pub/Sub models that necessitate a communication channel, NCache serves as a medium for topics. This entails the publisher dispatching messages to the designated topic and subscribers receiving notifications through the same topic. Employing NCache as the medium for topics promotes loose coupling within the model, offering enhanced abstraction and additional advantages for distributed topics. Publish The code snippet below initializes the messageService object using NCache MessagingService package. Initializing the Topic Java // Create a Topic in NCache. MessagingService messagingService = cache.getMessagingService(); Topic topic = messagingService.createTopic(topicName); // Create a thread pool for publishers ExecutorService publisherThreadPool = Executors.newFixedThreadPool(2); The below code snippet used to define register the subscribers to this topic Register subscribers to this Topic MessageReceivedListener subscriptionListener1 = new MessageReceivedListener() { @Override public void onMessageReceived(Object o, MessageEventArgs messageEventArgs) { messageReceivedSubscription1(messageEventArgs.getMessage()); } }; MessageReceivedListener subscriptionListener2 = new MessageReceivedListener() { @Override public void onMessageReceived(Object o, MessageEventArgs messageEventArgs) { messageReceivedSubscription2(messageEventArgs.getMessage()); } }; TopicSubscription subscription1 = topic.createSubscription(subscriptionListener1); TopicSubscription subscription2 = topic.createSubscription(subscriptionListener2); NCache provides two variants of durable subscriptions to cater to the message durability needs within your Java microservices: Shared Durable Subscriptions: This allows multiple subscribers to connect to a single subscription. The Round Robin approach is employed to distribute messages among the various subscribers. Even if a subscriber exits the network, messages persistently flow between the active subscribers. Exclusive Durable Subscriptions: In this type, only one active subscriber is allowed on a subscription at any given time. No new subscriber requests are accepted for the same subscription until the existing connection is active. Learn more Pub/Sub Messaging with NCache implementation here Pub/Sub Messaging in Cache: An Overview SQL Query on Cache NCache provides your microservices with the capability to perform SQL-like queries on indexed cache data. This functionality becomes particularly beneficial when the values of the keys storing the desired information are not known. It abstracts much of the lower-level cache API calls, contributing to clearer and more maintainable application code. This feature is especially advantageous for individuals who find SQL-like commands more intuitive and comfortable to work with. NCache provides functionality for searching and removing cache data through queries similar to SQL's SELECT and DELETE statements. However, operations like INSERT and UPDATE are not available. For executing SELECT queries within the cache, NCache utilizes ExecuteReader; the ExecuteScalar function is used to carry out a query and retrieve the first row's first column from the resulting data set, disregarding any extra columns or rows. For NCache SQL queries to function, indexes must be established on all objects undergoing search. This can be achieved through two methods: configuring the cache or utilizing code with "Custom Attributes" to annotate object fields. When objects are added to the cache, this approach automatically creates indexes on the specified fields. Code Snippet Java String cacheName = "demoCache"; // Connect to the cache and return a cache handle Cache cache = CacheManager.getCache(cacheName); // Adds all the products to the cache. This automatically creates indexes on various // attributes of Product object by using "Custom Attributes". addSampleData(cache); // $VALUE$ keyword means the entire object instead of individual attributes that are also possible String sql = "SELECT $VALUE$ FROM com.alachisoft.ncache.samples.Product WHERE category IN (?, ?) AND price < ?"; QueryCommand sqlCommand = new QueryCommand(sql); List<String> catParamList = new ArrayList<>(Arrays.asList(("Electronics"), ("Stationery"))); sqlCommand.getParameters().put("category", catParamList); sqlCommand.getParameters().put("price", 2000); // ExecuteReader returns ICacheReader with the query resultset CacheReader resultSet = cache.getSearchService().executeReader(sqlCommand); List<Product> fetchedProducts = new ArrayList<>(); if (resultSet.getFieldCount() > 0) { while (resultSet.read()) { // getValue() with $VALUE$ keyword returns the entire object instead of just one column fetchedProducts.add(resultSet.getValue("$VALUE$", Product.class)); } } printProducts(fetchedProducts); Utilize SQL in NCache to perform queries on cached data by focusing on object attributes and Tags, rather than solely relying on keys. In this example, we utilize "Custom Attributes" to generate an index on the Product object. Learn more about SQL Query with NCache in Java Query Data in Cache Using SQL Read-Thru and Write-Thru Utilize the Data Source Providers feature of NCache to position it as the primary interface for data access within your microservices architecture. When a microservice needs data, it should first query the cache. If the data is present, the cache supplies it directly. Otherwise, the cache employs a read-thru handler to fetch the data from the datastore on behalf of the client, caches it, and then provides it to the microservice. In a similar fashion, for write operations (such as Add, Update, Delete), a microservice can perform these actions on the cache. The cache then automatically carries out the corresponding write operation on the datastore using a write-thru handler. Furthermore, you have the option to compel the cache to fetch data directly from the datastore, regardless of the presence of a possibly outdated version in the cache. This feature is essential when microservices require the most current information and complements the previously mentioned cache consistency strategies. The integration of the Data Source Provider feature not only simplifies your application code but also, when combined with NCache's database synchronization capabilities, ensures that the cache is consistently updated with fresh data for processing. ReadThruProvider For implementing Read-Through caching, it's necessary to create an implementation of the ReadThruProvider interface in Java Here's a code snippet to get started with implementing Read-Thru in your microservices: Java ReadThruOptions readThruOptions = new ReadThruOptions(ReadMode.ReadThru, _readThruProviderName); product = _cache.get(_productId, readThruOptions, Product.class); Read more about Read-Thru implementation here: Read-Through Provider Configuration and Implementation WriteThruProvider: For implementing Write-Through caching, it's necessary to create an implementation of the WriteThruProvider interface in Java The code snippet to get started with implementing Write-Thru in your microservices: Java _product = new Product(); WriteThruOptions writeThruOptions = new WriteThruOptions(WriteMode.WriteThru, _writeThruProviderName) CacheItem cacheItem= new CacheItem(_customer) _cache.insert(_product.getProductID(), cacheItem, writeThruOptions); Read more about Write-Thru implementation here: Write-Through Provider Configuration and Implementation Summary Microservices are designed to be autonomous, enabling independent development, testing, and deployment from other microservices. While microservices provide benefits in scalability and rapid development cycles, some components of the application stack can present challenges. One such challenge is the use of relational databases, which may not support the necessary scale-out to handle growing loads. This is where a distributed caching solution like NCache becomes valuable. In this article, we have seen the variety of ready-to-use features like pub/sub messaging, data caching, SQL Query, Read-Thru and Write-Thru, and Hibernate second-level Java Cache techniques offered by NCache that simplify and streamline the integration of data caching into your microservices application, making it an effortless and natural extension.

By Gowtham K

Leveraging Java's Fork/Join Framework for Efficient Parallel Programming: Part 1

In concurrent programming, efficient parallelism is essential for maximizing the performance of applications. Java, being a popular programming language for various domains, provides robust support for parallel programming through its Fork/Join framework. This framework enables developers to write concurrent programs that leverage multicore processors effectively. In this comprehensive guide, we'll delve into the intricacies of the Fork/Join framework, explore its underlying principles, and provide practical examples to demonstrate its usage. Key Components ForkJoinPool: The central component of the Fork/Join Framework is ForkJoinPool, which manages a pool of worker threads responsible for executing tasks. It automatically scales the number of threads based on the available processors, optimizing resource utilization. ForkJoinTask: ForkJoinTaskis an abstract class representing a task that can be executed asynchronously. It provides two main subclasses: RecursiveTask: Used for tasks that return a result RecursiveAction: Used for tasks that don't return a result (i.e., void tasks) ForkJoinWorkerThread: This class represents worker threads within the ForkJoinPool. It provides hooks for customization, allowing developers to define thread-specific behavior. Deep Dive Into Fork/Join Workflow Task partitioning: When a task is submitted to the ForkJoinPool, it's initially executed sequentially until a certain threshold is reached. Beyond this threshold, the task is recursively split into smaller subtasks, which are distributed among the worker threads. Task execution: Worker threads execute the subtasks assigned to them in parallel. If a thread encounters a subtask marked for further division (i.e., "forked"), it splits the task and submits the subtasks to the pool. Result aggregation: Once the subtasks complete their execution, their results are combined to produce the final result. This process continues recursively until all subtasks are completed, and the final result is obtained. Take, for instance, a task designed to calculate the sum of values in an integer array. For small arrays, the task computes the sum directly. For larger arrays, it splits the array and assigns the subarrays to new tasks, which are then executed in parallel. Java class ArraySumCalculator extends RecursiveTask<Integer> { private int[] array; private int start, end; ArraySumCalculator(int[] array, int start, int end) { this.array = array; this.start = start; this.end = end; } @Override protected Integer compute() { if (end - start <= THRESHOLD) { int sum = 0; for (int i = start; i < end; i++) { sum += array[i]; } return sum; } else { int mid = start + (end - start) / 2; ArraySumCalculator leftTask = new ArraySumCalculator(array, start, mid); ArraySumCalculator rightTask = new ArraySumCalculator(array, mid, end); leftTask.fork(); int rightSum = rightTask.compute(); int leftSum = leftTask.join(); return leftSum + rightSum; } } } This task can then be executed by a ForkJoinPool: Java ForkJoinPool pool = new ForkJoinPool(); Integer totalSum = pool.invoke(new ArraySumCalculator(array, 0, array.length)); The Mechanics Behind ForkJoinPool The ForkJoinPool distinguishes itself as a specialized variant of ExecutorService, adept at managing a vast array of tasks, particularly those that adhere to the recursive nature of Fork/Join operations. Here's a breakdown of its fundamental components and operational dynamics: The Work-Stealing Paradigm Individual task queues: Every worker thread within a ForkJoinPool is equipped with its deque (double-ended queue) for tasks. Newly initiated tasks by a thread are placed at the head of its deque. Task redistribution: Threads that deplete their task queue engage in "stealing" tasks from the bottom of other threads' deques. This strategy of redistributing work ensures a more even workload distribution among threads, enhancing efficiency and resource utilization. ForkJoinTask Dynamics Task division: The act of forking divides a larger task into smaller, manageable subtasks, which are then dispatched to the pool for execution by available threads. This division places the subdivided tasks into the initiating thread's deque. Task completion: When a task awaits the completion of its forked subtasks (through the join method), it doesn't remain idle but instead seeks out other tasks to execute, either from its deque or by stealing, maintaining active participation in the pool's workload. Task Processing Logic Execution order: Worker threads typically process tasks in a last-in-first-out (LIFO) sequence, optimizing for tasks that are likely interconnected and could benefit from data locality. Conversely, the stealing process adheres to a first-in-first-out (FIFO) sequence, promoting a balanced task distribution. Adaptive Thread Management Responsive scaling: The ForkJoinPool dynamically adjusts its active thread count in response to the current workload and task characteristics, aiming to balance effective core utilization against the drawbacks of excessive threading, such as overhead and resource contention. Leveraging Internal Mechanics for Performance Optimization Grasping the inner workings of ForkJoinPool is essential for devising effective strategies for task granularity, pool configuration, and task organization: Determining task size: Understanding the individual task queues per thread can inform the decision-making process regarding the optimal task size, balancing between minimizing management overhead and ensuring full exploitation of the work-stealing feature. Tailoring ForkJoinPool settings: Insights into the pool's dynamic thread adjustment capabilities and work-stealing algorithm can guide the customization of pool parameters, such as parallelism levels, to suit specific application demands and hardware capabilities. Ensuring balanced workloads: Knowledge of how tasks are processed and redistributed can aid in structuring tasks to facilitate efficient workload distribution across threads, optimizing resource usage. Strategizing task design: Recognizing the impact of fork and join operations on task execution and thread engagement can lead to more effective task structuring, minimizing downtime, and maximizing parallel efficiency. Complex Use Cases For more complex scenarios, consider tasks that involve recursive data structures or algorithms, such as parallel quicksort or mergesort. These algorithms are inherently recursive and can benefit significantly from the Fork/Join framework's ability to handle nested tasks efficiently. For instance, in a parallel mergesort implementation, the array is divided into halves until the base case is reached. Each half is then sorted in parallel, and the results are merged. This approach can dramatically reduce sorting time for large datasets. Java class ParallelMergeSort extends RecursiveAction { private int[] array; private int start, end; ParallelMergeSort(int[] array, int start, int end) { this.array = array; this.start = start; this.end = end; } @Override protected void compute() { if (end - start <= THRESHOLD) { Arrays.sort(array, start, end); // Direct sort for small arrays } else { int mid = start + (end - start) / 2; ParallelMergeSort left = new ParallelMergeSort(array, start, mid); ParallelMergeSort right = new ParallelMergeSort(array, mid, end); invokeAll(left, right); // Concurrently sort both halves merge(array, start, mid, end); // Merge the sorted halves } } // Method to merge two halves of an array private void merge(int[] array, int start, int mid, int end) { // Implementation of merging logic } } Advanced Tips and Best Practices Dynamic Task Creation In scenarios where the data structure is irregular or the problem size varies significantly, dynamically creating tasks based on the runtime characteristics of the data can lead to more efficient utilization of system resources. Custom ForkJoinPool Management For applications running multiple Fork/Join tasks concurrently, consider creating separate ForkJoinPool instances with custom parameters to optimize the performance of different task types. This allows for fine-tuned control over thread allocation and task handling. Exception Handling Use the ForkJoinTask's get method, which throws an ExecutionException if any of the recursively executed tasks result in an exception. This approach allows for centralized exception handling, simplifying debugging, and error management. Java try { forkJoinPool.invoke(new ParallelMergeSort(array, 0, array.length)); } catch (ExecutionException e) { Throwable cause = e.getCause(); // Get the actual cause of the exception // Handle the exception appropriately } Workload Balancing When dealing with tasks of varying sizes, it's crucial to balance the workload among threads to avoid scenarios where some threads remain idle while others are overloaded. Techniques such as work stealing, as implemented by the Fork/Join framework, are essential in such cases. Avoiding Blocking When a task waits for another task to complete, it can lead to inefficiencies and reduced parallelism. Whenever possible, structure your tasks to minimize blocking operations. Utilizing the join method after initiating all forked tasks helps keep threads active. Performance Monitoring and Profiling Java's VisualVM or similar profiling tools can be invaluable in identifying performance bottlenecks and understanding how tasks are executed in parallel. Monitoring CPU usage, memory consumption, and task execution times helps pinpoint inefficiencies and guide optimizations. For instance, if VisualVM shows that most of the time is spent on a small number of tasks, it might indicate that the task granularity is too coarse, or that certain tasks are much more computationally intensive than others. Load Balancing and Work Stealing The Fork/Join framework's work-stealing algorithm is designed to keep all processor cores busy, but imbalances can still occur, especially with heterogeneous tasks. In such cases, breaking down tasks into smaller parts or using techniques to dynamically adjust the workload can help achieve better load balancing. An example strategy might involve monitoring task completion times and dynamically adjusting the size of future tasks based on this feedback, ensuring that all cores finish their workload at roughly the same time. Avoiding Common Pitfalls Common pitfalls such as unnecessary task splitting, improper use of blocking operations, or neglecting exceptions can degrade performance. Ensuring tasks are divided in a manner that maximizes parallel execution without creating too much overhead is key. Additionally, handling exceptions properly and avoiding blocking operations within tasks can prevent slowdowns and ensure smooth execution. Enhancing Performance With Strategic Tuning Through strategic tuning and optimization, developers can unleash the full potential of the Fork/Join framework, achieving remarkable improvements in the performance of parallel tasks. By carefully considering task granularity, customizing the Fork/JoinPool, diligently monitoring performance, and avoiding pitfalls, applications can be optimized to fully leverage the computational resources available, leading to faster, more efficient parallel processing. Conclusion The Fork/Join framework in Java offers a streamlined approach to parallel programming, abstracting complexities for developers. By mastering its components and inner workings, developers can unlock the full potential of multicore processors. With its intuitive design and efficient task management, the framework enables scalable and high-performance parallel applications. Armed with this understanding, developers can confidently tackle complex computational tasks, optimize performance, and meet the demands of modern computing environments. The Fork/Join framework remains a cornerstone of parallel programming in Java, empowering developers to harness the power of concurrency effectively.

By Andrei Tuchin

CORE

Advanced Brain-Computer Interfaces With Java

In the first part of this series, we introduced the basics of brain-computer interfaces (BCIs) and how Java can be employed in developing BCI applications. In this second part, let's delve deeper into advanced concepts and explore a real-world example of a BCI application using NeuroSky's MindWave Mobile headset and their Java SDK. Advanced Concepts in BCI Development Motor Imagery Classification: This involves the mental rehearsal of physical actions without actual execution. Advanced machine learning algorithms like deep learning models can significantly improve classification accuracy. Event-Related Potentials (ERPs): ERPs are specific patterns in brain signals that occur in response to particular events or stimuli. Developing BCI applications that exploit ERPs requires sophisticated signal processing techniques and accurate event detection algorithms. Hybrid BCI Systems: Hybrid BCI systems combine multiple signal acquisition methods or integrate BCIs with other physiological signals (like eye tracking or electromyography). Developing such systems requires expertise in multiple signal acquisition and processing techniques, as well as efficient integration of different modalities. Real-World BCI Example Developing a Java Application With NeuroSky's MindWave Mobile NeuroSky's MindWave Mobile is an EEG headset that measures brainwave signals and provides raw EEG data. The company provides a Java-based SDK called ThinkGear Connector (TGC), enabling developers to create custom applications that can receive and process the brainwave data. Step-by-Step Guide to Developing a Basic BCI Application Using the MindWave Mobile and TGC Establish Connection: Use the TGC's API to connect your Java application with the MindWave Mobile device over Bluetooth. The TGC provides straightforward methods for establishing and managing this connection. Java ThinkGearSocket neuroSocket = new ThinkGearSocket(this); neuroSocket.start(); Acquire Data: Once connected, your application will start receiving raw EEG data from the device. This data includes information about different types of brainwaves (e.g., alpha, beta, gamma), as well as attention and meditation levels. Java public void onRawDataReceived(int rawData) { // Process raw data } Process Data: Use signal processing techniques to filter out noise and extract useful features from the raw data. The TGC provides built-in methods for some basic processing tasks, but you may need to implement additional processing depending on your application's needs. Java public void onEEGPowerReceived(EEGPower eegPower) { // Process EEG power data } Interpret Data: Determine the user's mental state or intent based on the processed data. This could involve setting threshold levels for certain values or using machine learning algorithms to classify the data. For example, a high attention level might be interpreted as the user wanting to move a cursor on the screen. Java public void onAttentionReceived(int attention) { // Interpret attention data } Perform Action: Based on the interpretation of the data, have your application perform a specific action. This could be anything from moving a cursor, controlling a game character, or adjusting the difficulty level of a task. Java if (attention > ATTENTION_THRESHOLD) { // Perform action } Improving BCI Performance With Java Optimize Signal Processing: Enhance the quality of acquired brain signals by implementing advanced signal processing techniques, such as adaptive filtering or blind source separation. Employ Advanced Machine Learning Algorithms: Utilize state-of-the-art machine learning models, such as deep neural networks or ensemble methods, to improve classification accuracy and reduce user training time. Libraries like DeepLearning4j or TensorFlow Java can be employed for this purpose. Personalize BCI Models: Customize BCI models for individual users by incorporating user-specific features or adapting the model parameters during operation. This can be achieved using techniques like transfer learning or online learning. Implement Efficient Real-Time Processing: Ensure that your BCI application can process brain signals and generate output commands in real time. Optimize your code, use parallel processing techniques, and leverage Java's concurrency features to achieve low-latency performance. Evaluate and Validate Your BCI Application: Thoroughly test your BCI application on a diverse group of users and under various conditions to ensure its reliability and usability. Employ standard evaluation metrics and follow best practices for BCI validation. Conclusion Advanced BCI applications require a deep understanding of brain signal acquisition, processing, and classification techniques. Java, with its extensive libraries and robust performance, is an excellent choice for implementing such applications. By exploring advanced concepts, developing real-world examples, and continuously improving BCI performance, developers can contribute significantly to this revolutionary field.

By Arun Pandey

CORE

BigQuery DataFrames in Python

Google BigQuery is a powerful cloud-based data warehousing solution that enables users to analyze massive datasets quickly and efficiently. In Python, BigQuery DataFrames provide a Pythonic interface for interacting with BigQuery, allowing developers to leverage familiar tools and syntax for data querying and manipulation. In this comprehensive developer guide, we'll explore the usage of BigQuery DataFrames, their advantages, disadvantages, and potential performance issues. Introduction To BigQuery DataFrames BigQuery DataFrames serve as a bridge between Google BigQuery and Python, allowing seamless integration of BigQuery datasets into Python workflows. With BigQuery DataFrames, developers can use familiar libraries like Pandas to query, analyze, and manipulate BigQuery data. This Pythonic approach simplifies the development process and enhances productivity for data-driven applications. Advantages of BigQuery DataFrames Pythonic Interface: BigQuery DataFrames provide a Pythonic interface for interacting with BigQuery, enabling developers to use familiar Python syntax and libraries. Integration With Pandas: Being compatible with Pandas, BigQuery DataFrames allow developers to leverage the rich functionality of Pandas for data manipulation. Seamless Query Execution: BigQuery DataFrames handle the execution of SQL queries behind the scenes, abstracting away the complexities of query execution. Scalability: Leveraging the power of Google Cloud Platform, BigQuery DataFrames offer scalability to handle large datasets efficiently. Disadvantages of BigQuery DataFrames Limited Functionality: BigQuery DataFrames may lack certain advanced features and functionalities available in native BigQuery SQL. Data Transfer Costs: Transferring data between BigQuery and Python environments may incur data transfer costs, especially for large datasets. API Limitations: While BigQuery DataFrames provide a convenient interface, they may have limitations compared to directly using the BigQuery API for complex operations. Prerequisites Google Cloud Platform (GCP) Account: Ensure an active GCP account with BigQuery access. Python Environment: Set up a Python environment with the required libraries (pandas, pandas_gbq, and google-cloud-bigquery). Project Configuration: Configure your GCP project and authenticate your Python environment with the necessary credentials. Using BigQuery DataFrames Install Required Libraries Install the necessary libraries using pip: Python pip install pandas pandas-gbq google-cloud-bigquery Authenticate GCP Credentials Authenticate your GCP credentials to enable interaction with BigQuery: Python from google.auth import load_credentials # Load GCP credentials credentials, _ = load_credentials() Querying BigQuery DataFrames Use pandas_gbq to execute SQL queries and retrieve results as a DataFrame: Python import pandas_gbq # SQL Query query = "SELECT * FROM `your_project_id.your_dataset_id.your_table_id`" # Execute Query and Retrieve DataFrame df = pandas_gbq.read_gbq(query, project_id="your_project_id", credentials=credentials) Writing to BigQuery Write a DataFrame to a BigQuery table using pandas_gbq: Python # Write DataFrame to BigQuery pandas_gbq.to_gbq(df, destination_table="your_project_id.your_dataset_id.your_new_table", project_id="your_project_id", if_exists="replace", credentials=credentials) Advanced Features SQL Parameters Pass parameters to your SQL queries dynamically: Python params = {"param_name": "param_value"} query = "SELECT * FROM `your_project_id.your_dataset_id.your_table_id` WHERE column_name = @param_name" df = pandas_gbq.read_gbq(query, project_id="your_project_id", credentials=credentials, dialect="standard", parameters=params) Schema Customization Customize the DataFrame schema during the write operation: Python schema = [{"name": "column_name", "type": "INTEGER"}, {"name": "another_column", "type": "STRING"}] pandas_gbq.to_gbq(df, destination_table="your_project_id.your_dataset_id.your_custom_table", project_id="your_project_id", if_exists="replace", credentials=credentials, table_schema=schema) Performance Considerations Data Volume: Performance may degrade with large datasets, especially when processing and transferring data between BigQuery and Python environments. Query Complexity: Complex SQL queries may lead to longer execution times, impacting overall performance. Network Latency: Network latency between the Python environment and BigQuery servers can affect query execution time, especially for remote connections. Best Practices for Performance Optimization Use Query Filters: Apply filters to SQL queries to reduce the amount of data transferred between BigQuery and Python. Optimize SQL Queries: Write efficient SQL queries to minimize query execution time and reduce resource consumption. Cache Query Results: Cache query results in BigQuery to avoid re-executing queries for repeated requests. Conclusion BigQuery DataFrames offer a convenient and Pythonic way to interact with Google BigQuery, providing developers with flexibility and ease of use. While they offer several advantages, developers should be aware of potential limitations and performance considerations. By following best practices and optimizing query execution, developers can harness the full potential of BigQuery DataFrames for data analysis and manipulation in Python.

By Sreenath Devineni

Unleashing the Power of Java Interfaces

Java interfaces, for a very long time, were just that — interfaces, an anemic set of function prototypes. Even then, there were non-standard uses of interfaces (for example, marker interfaces), but that's it. However, since Java 8, there have been substantial changes in the interfaces. Additions of default and static methods enabled many new possibilities. For example, enabled adding of new functionality to existing interfaces without breaking old code. Or hiding all implementations behind factory methods, enforcing the “code against interface” policy. The addition of sealed interfaces enabled the creation of true sum types and expressions in code design intents. Together, these changes made Java interfaces a powerful, concise, and expressive tool. Let’s take a look at some non-traditional applications of Java interfaces. Fluent Builder Fluent (or Staged) Builder is a pattern used to assemble object instances. Unlike the traditional Builder pattern, it prevents the creation of incomplete objects and enforces a fixed order of field initialization. These properties make it the preferred choice for reliable and maintainable code. The idea behind Fluent Builder is rather simple. Instead of returning the same Builder instance after setting a property, it returns a new type (class or interface), which has only one method, therefore guiding the developer through the process of instance initialization. A fluent builder may omit the build() method at the end; for instance, assembling ends once the last field is set. Unfortunately, the straightforward implementation of Fluent Builder is very verbose: Java public record NameAge(String firstName, String lastName, Option<String> middleName, int age) { public static NameAgeBuilderStage1 builder() { return new NameAgeBuilder(); } public static class NameAgeBuilder implements NameAgeBuilderStage1, NameAgeBuilderStage2, NameAgeBuilderStage3, NameAgeBuilderStage4 { private String firstName; private String lastName; private Option<String> middleName; @Override public NameAgeBuilderStage2 firstName(String firstName) { this.firstName = firstName; return this; } @Override public NameAgeBuilderStage3 lastName(String lastName) { this.lastName = lastName; return this; } @Override public NameAgeBuilderStage4 middleName(Option<String> middleName) { this.middleName = middleName; return this; } @Override public NameAge age(int age) { return new NameAge(firstName, lastName, middleName, age); } } public interface NameAgeBuilderStage1 { NameAgeBuilderStage2 firstName(String firstName); } public interface NameAgeBuilderStage2 { NameAgeBuilderStage3 lastName(String lastName); } public interface NameAgeBuilderStage3 { NameAgeBuilderStage4 middleName(Option<String> middleName); } public interface NameAgeBuilderStage4 { NameAge age(int age); } } It is also not very safe, as it is still possible to cast the returned interface to NameAgeBuilder and call the age() method, getting an incomplete object. We might notice that each interface is a typical functional interface with only one method inside. With this in mind, we may rewrite the code above into the following: Java public record NameAge(String firstName, String lastName, Option<String> middleName, int age) { static NameAgeBuilderStage1 builder() { return firstName -> lastName -> middleName -> age -> new NameAge(firstName, lastName, middleName, age); } public interface NameAgeBuilderStage1 { NameAgeBuilderStage2 firstName(String firstName); } public interface NameAgeBuilderStage2 { NameAgeBuilderStage3 lastName(String lastName); } public interface NameAgeBuilderStage3 { NameAgeBuilderStage4 middleName(Option<String> middleName); } public interface NameAgeBuilderStage4 { NameAge age(int age); } } Besides being much more concise, this version is not susceptible to (even hacky) premature object creation. Reduction of Implementation Although default methods were created to enable the extension of existing interfaces without breaking the existing implementation, this is not the only use for them. For a long time, if we needed multiple implementations of the same interface, where many implementations share some code, the only way to avoid code duplication was to create an abstract class and inherit those implementations from it. Although this avoided code duplication, this solution is relatively verbose and causes unnecessary coupling. The abstract class is a purely technical entity that has no corresponding part in the application domain. With default methods, abstract classes are no longer necessary; common functionality can be written directly in the interface, reducing boilerplate, eliminating coupling, and improving maintainability. But what if we go further? Sometimes, it is possible to express all necessary functionality using only very few implementation-specific methods. Ideally — just one. This makes implementation classes very compact and easy to reason about and maintain. Let’s, for example, implement Maybe<T> monad (yet another name for Optional<T>/Option<T>). No matter how rich and diverse API we’re planning to implement, it still could be expressed as a call to a single method, let’s call it fold(): Java <R> R fold(Supplier<? extends R> nothingMapper, Function<? super T, ? extends R> justMapper) This method accepts two functions; one is called when the value is present and another when the value is missing. The result of the application is just returned as the result of the implemented method. With this method, we can implement map() and flatMap() as: Java default Maybe map(Function<? super T, U> mapper) { return fold(Maybe::nothing, t -> just(mapper.apply(t))); } default Maybe flatMap(Function<? super T, Maybe> mapper) { return fold(Maybe::nothing, mapper); } These implementations are universal and applicable to both variants. Note that since we have exactly two implementations, it makes perfect sense to make the interface sealed. And to even further reduce the amount of boilerplate — use records: Java public sealed interface Maybe<T> { default Maybe map(Function<? super T, U> mapper) { return fold(Maybe::nothing, t -> just(mapper.apply(t))); } default Maybe flatMap(Function<? super T, Maybe> mapper) { return fold(Maybe::nothing, mapper); } <R> R fold(Supplier<? extends R> nothingMapper, Function<? super T, ? extends R> justMapper); static <T> Just<T> just(T value) { return new Just<>(value); } @SuppressWarnings("unchecked") static <T> Nothing<T> nothing() { return (Nothing<T>) Nothing.INSTANCE; } static <T> Maybe<T> maybe(T value) { return value == null ? nothing() : just(value); } record Just<T>(T value) implements Maybe<T> { public <R> R fold(Supplier<? extends R> nothingMapper, Function<? super T, ? extends R> justMapper) { return justMapper.apply(value); } } record Nothing<T>() implements Maybe<T> { static final Nothing<?> INSTANCE = new Nothing<>(); @Override public <R> R fold(Supplier<? extends R> nothingMapper, Function<? super T, ? extends R> justMapper) { return nothingMapper.get(); } } } Although this is not strictly necessary for demonstration, this implementation uses a shared constant for the implementation of 'Nothing', reducing allocation. Another interesting property of this implementation — it uses no if statement (nor ternary operator) for the logic. This improves performance and enables better optimization by the Java compiler. Another useful property of this implementation — it is convenient for pattern matching (unlike Java 'Optional'for example): Java var result = switch (maybe) { case Just<String>(var value) -> value; case Nothing<String> nothing -> "Nothing"; }; But sometimes, even implementation classes are not necessary. The example below shows how the entire implementation fits into the interface (full code can be found here): Java public interface ShortenedUrlRepository { default Promise<ShortenedUrl> create(ShortenedUrl shortenedUrl) { return QRY."INSERT INTO shortenedurl (\{template().fieldNames()}) VALUES (\{template().fieldValues(shortenedUrl)}) RETURNING *" .in(db()) .asSingle(template()); } default Promise<ShortenedUrl> read(String id) { return QRY."SELECT * FROM shortenedurl WHERE id = \{id}" .in(db()) .asSingle(template()); } default Promise<Unit> delete(String id) { return QRY."DELETE FROM shortenedurl WHERE id = \{id}" .in(db()) .asUnit(); } DbEnv db(); } To turn this interface into a working instance, all we need is to provide an instance of the environment. For example, like this: Java var dbEnv = DbEnv.with(dbEnvConfig); ShortenedUrlRepository repository = () -> dbEnv; This approach sometimes results in code that is too concise and sometimes requires writing a more verbose version to preserve context. I’d say that this is quite an unusual property for Java code, which is often blamed for verbosity. Utility … Interfaces? Well, utility (as well as constant) interfaces were not feasible for a long time. Perhaps the main reason is that such interfaces could be implemented, and constants, as well as utility functions, would be (unnecessary) part of the implementation. But with sealed interfaces, this issue can be solved in a way similar to how instantiation of utility classes is prevented: Java public sealed interface Utility { ... record unused() implements Utility {} } At first look, it makes no big sense to use this approach. However, the use of an interface eliminates the need for visibility modifiers for each method and/or constant. This, in turn, reduces the amount of syntactic noise, which is mandatory for classes but redundant for interfaces, as they have all their members public. Interfaces and Private Records The combination of these two constructs enables convenient writing code in “OO without classes” style, enforcing “code against interface” while reducing boilerplate at the same time. For example: Java public interface ContentType { String headerText(); ContentCategory category(); static ContentType custom(String headerText, ContentCategory category) { record contentType(String headerText, ContentCategory category) implements ContentType {} return new contentType(headerText, category); } } The private record serves two purposes: It keeps the use of implementation under complete control. No direct instantiations are possible, only via the static factory method. Keeps implementation close to the interface, simplifying support, extension, and maintenance. Note that the interface is not sealed, so one can do, for example, the following: Java public enum CommonContentTypes implements ContentType { TEXT_PLAIN("text/plain; charset=UTF-8", ContentCategory.PLAIN_TEXT), APPLICATION_JSON("application/json; charset=UTF-8", ContentCategory.JSON), ; private final String headerText; private final ContentCategory category; CommonContentTypes(String headerText, ContentCategory category) { this.headerText = headerText; this.category = category; } @Override public String headerText() { return headerText; } @Override public ContentCategory category() { return category; } } Conclusion Interfaces are a powerful Java feature, often underestimated and underutilized. This article is an attempt to shed light on the possible ways to utilize their power and get clean, expressive, concise, yet readable code.

By Sergiy Yevtushenko

Spring Boot 3.2: Replace Your RestTemplate With RestClient

In the world of Spring Boot, making HTTP requests to external services is a common task. Traditionally, developers have relied on RestTemplate for this purpose. However, with the evolution of the Spring Framework, a new and more powerful way to handle HTTP requests has emerged: the WebClient. In Spring Boot 3.2, a new addition called RestClient builds upon WebClient, providing a more intuitive and modern approach to consuming RESTful services. Origins of RestTemplate RestTemplate has been a staple in the Spring ecosystem for years. It's a synchronous client for making HTTP requests and processing responses. With RestTemplate, developers could easily interact with RESTful APIs using familiar Java syntax. However, as applications became more asynchronous and non-blocking, the limitations of RestTemplate started to become apparent. Here's a basic example of using RestTemplate to fetch data from an external API: Java var restTemplate = new RestTemplate(); var response = restTemplate.getForObject("https://api.example.com/data", String.class); System.out.println(response); Introduction of WebClient With the advent of Spring WebFlux, an asynchronous, non-blocking web framework, WebClient was introduced as a modern alternative to RestTemplate. WebClient embraces reactive principles, making it well-suited for building reactive applications. It offers support for both synchronous and asynchronous communication, along with a fluent API for composing requests. Here's how you would use WebClient to achieve the same HTTP request: Java var webClient = WebClient.create(); var response = webClient.get() .uri("https://api.example.com/data") .retrieve() .bodyToMono(String.class); response.subscribe(System.out::println); Enter RestClient in Spring Boot 3.2 Spring Boot 3.2 brings RestClient, a higher-level abstraction built on top of WebClient. RestClient simplifies the process of making HTTP requests even further by providing a more intuitive fluent API and reducing boilerplate code. It retains all the capabilities of WebClient while offering a more developer-friendly interface. Let's take a look at how RestClient can be used: var response = restClient .get() .uri(cepURL) .retrieve() .toEntity(String.class); System.out.println(response.getBody()); With RestClient, the code becomes more concise and readable. The RestClient handles the creation of WebClient instances internally, abstracting away the complexities of setting up and managing HTTP clients. Comparing RestClient With RestTemplate Let's compare RestClient with RestTemplate by looking at some common scenarios: Create RestTemplate: var response = new RestTemplate(); RestClient: var response = RestClient.create(); Or we can use our old RestTemplate as well: var myOldRestTemplate = new RestTemplate(); var response = RestClient.builder(myOldRestTemplate); GET Request RestTemplate: Java var response = restTemplate.getForObject("https://api.example.com/data", String.class); RestClient: var response = restClient .get() .uri(cepURL) .retrieve() .toEntity(String.class); POST Request RestTemplate: Java ResponseEntity<String> response = restTemplate.postForEntity("https://api.example.com/data", request, String.class); RestClient: var response = restClient .post() .uri("https://api.example.com/data") .body(request) .retrieve() .toEntity(String.class); Error Handling RestTemplate: Java try { String response = restTemplate.getForObject("https://api.example.com/data", String.class); } catch (RestClientException ex) { // Handle exception } RestClient: String request = restClient.get() .uri("https://api.example.com/this-url-does-not-exist") .retrieve() .onStatus(HttpStatusCode::is4xxClientError, (request, response) -> { throw new MyCustomRuntimeException(response.getStatusCode(), response.getHeaders()) }) .body(String.class); As seen in these examples, RestClient offers a more streamlined approach to making HTTP requests compared to RestTemplate. Spring Documentation gives us many other examples. Conclusion In Spring Boot 3.2, RestClient emerges as a modern replacement for RestTemplate, offering a more intuitive and concise way to consume RESTful services. Built on top of WebClient, RestClient embraces reactive principles while simplifying the process of making HTTP requests. Developers can now enjoy improved productivity and cleaner code when interacting with external APIs in their Spring Boot applications. It's recommended to transition from RestTemplate to RestClient for a more efficient and future-proof codebase.

By Fernando Boaglio

Be Punctual! Avoiding Kotlin’s lateinit In Spring Boot Testing

A sign of a good understanding of a programming language is not whether one is simply knowledgeable about the language’s functionality, but why such functionality exists. Without knowing this “why," the developer runs the risk of using functionality in situations where its use might not be ideal - or even should be avoided in its entirety! The case in point for this article is the lateinit keyword in Kotlin. Its presence in the programming language is more or less a way to resolve what would otherwise be contradictory goals for Kotlin: Maintain compatibility with existing Java code and make it easy to transcribe from Java to Kotlin. If Kotlin were too dissimilar to Java - and if the interaction between Kotlin and Java code bases were too much of a hassle - then adoption of the language might have never taken off. Prevent developers from declaring class members without explicitly declaring their value, either directly or via constructors. In Java, doing so will assign a default value, and this leaves non-primitives - which are assigned a null value - at the risk of provoking a NullPointerException if they are accessed without a value being provided beforehand. The problem here is this: what happens when it’s impossible to declare a class field’s value immediately? Take, for example, the extension model in the JUnit 5 testing framework. Extensions are a tool for creating reusable code that conducts setup and cleanup actions before and after the execution of each or all tests. Below is an example of an extension whose purpose is to clear out all designated database tables after the execution of each test via a Spring bean that serves as the database interface: Java public class DBExtension implements BeforeAllCallback, AfterEachCallback { private NamedParameterJdbcOperations jdbcOperations; @Override public void beforeAll(ExtensionContext extensionContext) { jdbcOperations = SpringExtension.getApplicationContext(extensionContext) .getBean(NamedParameterJdbcTemplate.class); clearDB(); } @Override public void afterEach(ExtensionContext extensionContext) throws Exception { clearDB(); } private void clearDB() { Stream.of("table_one", "table_two", "table_three").forEach((tableName) -> jdbcOperations.update("TRUNCATE " + tableName, new MapSqlParameterSource()) ); } } (NOTE: Yes, using the @Transactional annotation is possible for tests using Spring Boot tests that conduct database transactions, but some use cases make automated transaction roll-backs impossible; for example, when a separate thread is spawned to execute the code for the database interactions.) Given that the field jdbcOperations relies on the Spring framework loading the proper database interface bean when the application is loaded, it cannot be assigned any substantial value upon declaration. Thus, it receives an implicit default value of null until the beforeAll() function is executed. As described above, this approach is forbidden in Kotlin, so the developer has three options: Declare jdbcOperations as var, assign a garbage value to it in its declaration, then assign the “real” value to the field in beforeAll(): Kotlin class DBExtension : BeforeAllCallback, AfterEachCallback { private var jdbcOperations: NamedParameterJdbcOperations = StubJdbcOperations() override fun beforeAll(extensionContext: ExtensionContext) { jdbcOperations = SpringExtension.getApplicationContext(extensionContext) .getBean(NamedParameterJdbcOperations::class.java) clearDB() } override fun afterEach(extensionContext: ExtensionContext) { clearDB() } private fun clearDB() { listOf("table_one", "table_two", "table_three").forEach { tableName: String -> jdbcOperations.update("TRUNCATE $tableName", MapSqlParameterSource()) } } } The downside here is that there’s no check for whether the field has been assigned the “real” value, running the risk of invalid behavior when the field is accessed if the “real” value hasn’t been assigned for whatever reason. 2. Declare jdbcOperations as nullable and assign null to the field, after which the field will be assigned its “real” value in beforeAll(): Kotlin class DBExtension : BeforeAllCallback, AfterEachCallback { private var jdbcOperations: NamedParameterJdbcOperations? = null override fun beforeAll(extensionContext: ExtensionContext) { jdbcOperations = SpringExtension.getApplicationContext(extensionContext) .getBean(NamedParameterJdbcOperations::class.java) clearDB() } override fun afterEach(extensionContext: ExtensionContext) { clearDB() } private fun clearDB() { listOf("table_one", "table_two", "table_three").forEach { tableName: String -> jdbcOperations!!.update("TRUNCATE $tableName", MapSqlParameterSource()) } } } The downside here is that declaring the field as nullable is permanent; there’s no mechanism to declare a type as nullable “only” until its value has been assigned elsewhere. Thus, this approach forces the developer to force the non-nullable conversion whenever accessing the field, in this case using the double-bang (i.e. !!) operator to access the field’s update() function. 3. Utilize the lateinit keyword to postpone a value assignment to jdbcOperations until the execution of the beforeAll() function: Kotlin class DBExtension : BeforeAllCallback, AfterEachCallback { private lateinit var jdbcOperations: NamedParameterJdbcOperations override fun beforeAll(extensionContext: ExtensionContext) { jdbcOperations = SpringExtension.getApplicationContext(extensionContext) .getBean(NamedParameterJdbcOperations::class.java) clearDB() } override fun afterEach(extensionContext: ExtensionContext) { clearDB() } private fun clearDB() { listOf("table_one", "table_two", "table_three").forEach { tableName: String -> jdbcOperations.update("TRUNCATE $tableName", MapSqlParameterSource()) } } } No more worrying about silently invalid behavior or being forced to “de-nullify” the field each time it’s being accessed! The “catch” is that there’s still no compile-time mechanism for determining whether the field has been accessed before it’s been assigned a value - it’s done at run-time, as can be seen when decompiling the clearDB() function: Java private final void clearDB() { Iterable $this$forEach$iv = (Iterable)CollectionsKt.listOf(new String[]{"table_one", "table_two", "table_three"}); int $i$f$forEach = false; NamedParameterJdbcOperations var10000; String tableName; for(Iterator var3 = $this$forEach$iv.iterator(); var3.hasNext(); var10000.update("TRUNCATE " + tableName, (SqlParameterSource)(new MapSqlParameterSource()))) { Object element$iv = var3.next(); tableName = (String)element$iv; int var6 = false; var10000 = this.jdbcOperations; if (var10000 == null) { Intrinsics.throwUninitializedPropertyAccessException("jdbcOperations"); } } } Not ideal, considering what’s arguably Kotlin’s star feature (compile-time checking of variable nullability to reduce the likelihood of the “Billion-Dollar Mistake”) - but again, it’s a “least-worst” compromise to bridge the gap between Kotlin code and the Java-based code that provides no alternatives that adhere to Kotlin’s design philosophy. Use Wisely! Aside from the above-mentioned issue of conducting null checks only at run-time instead of compile-time, lateinit possesses a few more drawbacks: A field that uses lateinit cannot be an immutable val, as its value is being assigned at some point after the field’s declaration, so the field is exposed to the risk of inadvertently being modified at some point by an unwitting developer and causing logic errors. Because the field is not instantiated upon declaration, any other fields that rely on this field - be it via some function call to the field or passing it in as an argument to a constructor - cannot be instantiated upon declaration as well. This makes lateinit a bit of a “viral” feature: using it on field A forces other fields that rely on field A to use lateinit as well. Given that this mutability of lateinit fields goes against another one of Kotlin’s guiding principles - make fields and variables immutable where possible (for example, function arguments are completely immutable) to avoid logic errors by mutating a field/variable that shouldn’t have been changed - its use should be restricted to where no alternatives exist. Unfortunately, several code patterns that are prevalent in Spring Boot and Mockito - and likely elsewhere, but that’s outside the scope of this article - were built on Java’s tendency to permit uninstantiated field declarations. This is where the ease of transcribing Java code to Kotlin code becomes a double-edged sword: it’s easy to simply move the Java code over to a Kotlin file, slap the lateinit keyword on a field that hasn’t been directly instantiated in the Java code, and call it a day. Take, for instance, a test class that: Auto-wires a bean that’s been registered in the Spring Boot component ecosystem Injects a configuration value that’s been loaded in the Spring Boot environment Mocks a field’s value and then passes said mock into another field’s object Creates an argument captor for validating arguments that are passed to specified functions during the execution of one or more test cases Instantiates a mocked version of a bean that has been registered in the Spring Boot component ecosystem and passes it to a field in the test class Here is the code for all of these points put together: Kotlin @SpringBootTest @ExtendWith(MockitoExtension::class) @AutoConfigureMockMvc class FooTest { @Autowired private lateinit var mockMvc: MockMvc @Value("\${foo.value}") private lateinit var fooValue: String @Mock private lateinit var otherFooRepo: OtherFooRepo @InjectMocks private lateinit var otherFooService: OtherFooService @Captor private lateinit var timestampCaptor: ArgumentCaptor<Long> @MockBean private lateinit var fooRepo: FooRepo // Tests below } A better world is possible! Here are ways to avoid each of these constructs so that one can write “good” idiomatic Kotlin code while still retaining the use of auto wiring, object mocking, and argument capturing in the tests. Becoming “Punctual” Note: The code in these examples uses Java 17, Kotlin 1.9.21, Spring Boot 3.2.0, and Mockito 5.7.0. @Autowired/@Value Both of these constructs originate in the historic practice of having Spring Boot inject the values for the fields in question after their containing class has been initialized. This practice has since been deprecated in favor of declaring the values that are to be injected into the fields as arguments for the class’s constructor. For example, this code follows the old practice: Kotlin @Service class FooService { @Autowired private lateinit var fooRepo: FooRepo @Value("\${foo.value}") private lateinit var fooValue: String } It can be updated to this code: Kotlin @Service class FooService( private val fooRepo: FooRepo, @Value("\${foo.value}") private val fooValue: String, ) { } Note that aside from being able to use the val keyword, the @Autowired annotation can be removed from the declaration of fooRepo as well, as the Spring Boot injection mechanism is smart enough to recognize that fooRepo refers to a bean that can be instantiated and passed in automatically. Omitting the @Autowired annotation isn’t possible for testing code: test files aren't actually a part of the Spring Boot component ecosystem, and thus, won’t know by default that they need to rely on the auto-wired resource injection system - but otherwise, the pattern is the same: Kotlin @SpringBootTest @ExtendWith(MockitoExtension::class) @AutoConfigureMockMvc class FooTest( @Autowired private val mockMvc: MockMvc, @Value("\${foo.value}") private val fooValue: String, ) { @Mock private lateinit var otherFooRepo: OtherFooRepo @InjectMocks private lateinit var otherFooService: OtherFooService @Captor private lateinit var timestampCaptor: ArgumentCaptor<Long> @MockBean private lateinit var fooRepo: FooRepo // Tests below } @Mock/@InjectMocks The Mockito extension for JUnit allows a developer to declare a mock object and leave the actual mock instantiation and resetting of the mock’s behavior - as well as injecting these mocks into the dependent objects like otherFooService in the example code - to the code within MockitoExtension. Aside from the disadvantages mentioned above about being forced to use mutable objects, it poses quite a bit of “magic” around the lifecycle of the mocked objects that can be easily avoided by directly instantiating and manipulating the behavior of said objects: Kotlin @SpringBootTest @ExtendWith(MockitoExtension::class) @AutoConfigureMockMvc class FooTest( @Autowired private val mockMvc: MockMvc, @Value("\${foo.value}") private val fooValue: String, ) { private val otherFooRepo: OtherFooRepo = mock() private val otherFooService = OtherFooService(otherFooRepo) @Captor private lateinit var timestampCaptor: ArgumentCaptor<Long> @MockBean private lateinit var fooRepo: FooRepo @AfterEach fun afterEach() { reset(otherFooRepo) } // Tests below } As can be seen above, a post-execution hook is now necessary to clean up the mocked object otherFooRepo after the test execution(s), but this drawback is more than made up for by otherfooRepo and otherFooService now being immutable as well as having complete control over both objects’ lifetimes. @Captor Just as with the @Mock annotation, it’s possible to remove the @Captor annotation from the argument captor and declare its value directly in the code: Kotlin @SpringBootTest @AutoConfigureMockMvc class FooTest( @Autowired private val mockMvc: MockMvc, @Value("\${foo.value}") private val fooValue: String, ) { private val otherFooRepo: OtherFooRepo = mock() private val otherFooService = OtherFooService(otherFooRepo) private val timestampCaptor: ArgumentCaptor<Long> = ArgumentCaptor.captor() @MockBean private lateinit var fooRepo: FooRepo @AfterEach fun afterEach() { reset(otherFooRepo) } // Tests below } While there’s a downside in that there’s no mechanism in resetting the argument captor after each test (meaning that a call to getAllValues() would return artifacts from other test cases’ executions), there’s the case to be made that an argument captor could be instantiated as an object within only the test cases where it is to be used and done away with using an argument captor as a test class’s field. In any case, now that both @Mock and @Captor have been removed, it’s possible to remove the Mockito extension as well. @MockBean A caveat here: the use of mock beans in Spring Boot tests could be considered a code smell, signaling that, among other possible issues, the IO layer of the application isn’t being properly controlled for integration tests, that the test is de-facto a unit test and should be rewritten as such, etc. Furthermore, too much usage of mocked beans in different arrangements can cause test execution times to spike. Nonetheless, if it’s absolutely necessary to use mocked beans in the tests, a solution does exist for converting them into immutable objects. As it turns out, the @MockBean annotation can be used not just on field declarations, but also for class declarations as well. Furthermore, when used at the class level, it’s possible to pass in the classes that are to be declared as mock beans for the test in the value array for the annotation. This results in the mock bean now being eligible to be declared as an @Autowired bean just like any “normal” Spring Boot bean being passed to a test class: Kotlin @SpringBootTest @AutoConfigureMockMvc @MockBean(value = [FooRepo::class]) class FooTest( @Autowired private val mockMvc: MockMvc, @Value("\${foo.value}") private val fooValue: String, @Autowired private val fooRepo: FooRepo, ) { private val otherFooRepo: OtherFooRepo = mock() private val otherFooService = OtherFooService(otherFooRepo) private val timestampCaptor: ArgumentCaptor<Long> = ArgumentCaptor.captor() @AfterEach fun afterEach() { reset(fooRepo, otherFooRepo) } // Tests below } Note that like otherFooRepo, the object will have to be reset in the cleanup hook. Also, there’s no indication that fooRepo is a mocked object as it’s being passed to the constructor of the test class, so writing patterns like declaring all mocked beans in an abstract class and then passing them to specific extending test classes when needed runs the risk of “out of sight, out of mind” in that the knowledge that the bean is mocked is not inherently evident. Furthermore, better alternatives to mocking beans exist (for example, WireMock and Testcontainers) to handle mocking out the behavior of external components. Conclusion Note that each of these techniques is possible for code written in Java as well and provides the very same benefits of immutability and control of the objects’ lifecycles. What makes these recommendations even more pertinent to Kotlin is that they allow the user to align more closely with Kotlin’s design philosophy. Kotlin isn’t simply “Java with better typing:" It’s a programming language that places an emphasis on reducing common programming errors like accidentally accessing null pointers as well as items like inadvertently re-assigning objects and other pitfalls. Going beyond merely looking up the tools that are at one’s disposal in Kotlin to finding out why they’re available in the form that they exist will yield dividends of much higher productivity in the language, less risks of trying to fight against the language instead of focusing on solving the tasks at hand, and, quite possibly, making writing code in the language an even more rewarding and fun experience.

By Severn Everett

Coding

Functions of Coding

Frameworks

Java

JavaScript

Languages

Tools

DZone's Featured Coding Resources

The Latest Coding Topics