% vim: set ts=2 sw=2 et tw=80: \documentclass[11pt,a4paper]{scrartcl} \usepackage[utf8]{inputenc} \usepackage[margin=2.25cm]{geometry} \usepackage{hyperref} \usepackage{listings} \usepackage{xcolor} \usepackage{lmodern} \usepackage{booktabs} \usepackage{graphicx} \usepackage{float} \usepackage{tikz} \usepackage{listings} \usepackage{pgfplots} \usepackage{subcaption} \setlength{\parindent}{0cm} \setlength{\parskip}{0.3em} \hypersetup{pdfborder={0 0 0}} \usepackage[nomessages]{fp} \renewcommand*{\titlepagestyle}{empty} \lstset{ basicstyle=\small\ttfamily, %frame=shadowbox, rulesepcolor=\color{black}, columns=fullflexible, commentstyle=\color{gray}, keywordstyle=\color{blue}, mathescape=true, aboveskip=1em, captionpos=b, abovecaptionskip=1em, belowcaptionskip=1em } \title{Assginment 1 -- Software Design and Modelling} \author{Volodymyr Karpenko \and Claudio Maggioni} \begin{document} \begin{titlepage} \maketitle \pagenumbering{roman} \tableofcontents \lstlistoflistings \end{titlepage} \section{Project selection process} \pagenumbering{arabic} We have to choose a Java-based project on GitHub that follows the following requirements: \begin{itemize} \item 100 or more stars; \item 100 or more forks; \item 10 or more open issues; \item 50.000 or more lines of code. \end{itemize} Additionally, we added some less strict constraints that we thought would lead to a more significant and influential analysis: \begin{itemize} \item There must be evidence that the project follows business-oriented conventions. This excludes amateur or personal projects that might have fewer design pattern applications due to their nature. \item Repository data, documentation, and comments must be written in English. Many repositories that are at the top of the search results provided by the hard requirements are not in English, and this drastically hampers our ability to understand the code; \item The artifact the project produces must not rely on external components and have a streamlined build process, with all code stored in a single Maven/Gradle module. This improves our ability to tinker with the project more quickly and the pattern detection process, which requires all \textit{.class} files related to the project to be stored in a single directory tree. \end{itemize} Additionally, instead of querying GitHub directly for projects, we decided to see if libraries we knew already in our Java development career would match the hard and soft requirements we set for ourselves. Therefore, we considered the following GitHub repositories: \begin{description} \item[vavr-io/vavr] a Java library for functional programming, was discarded as the project is less than 20.000 lines of code and does not meet the rigid requirements; \item[bitcoin4j/bitcoin4j] a Java implementation of the bitcoin protocol, discarded as the project is distributed in several subprojects, and therefore the build process is nontrivial; \item[FasterXML/jackson-core] is the core ''module`` of a Java JSON serialization and deserialization library. We chose this project because it meets the selection criteria. It does not rely on external components for its execution. Finally, the project structure uses a single Maven module for its sources and is thus easy to analyze. \end{description} \subsection {The Jackson Core Project} As mentioned, Jackson is a library that offers serialization and deserialization capabilities in JSON format. It is highly extensible and customizable through a robust but flexible API. The library is divided into what the Jackson developers call “modules,” i.e., plug-ins that can augment the serialization and deserialization process. Some modules, like the \textit{jackson-dataformat-xml} module, target different encoding languages like XML. The chosen repository contains only the \textit{core} module of Jackson. The \textit{core} module implements the necessary library abstractions and interfaces to allow other modules to be plugged-in. Additionally, the \textit{core} module implements the tokenizer and low-level abstractions to work with the JSON format. We will refer to this module as ``Jackson'' or ``Jackson Core'' interchangeably throughout this report. We choose to analyze version 2.13.4 of the module (i.e.\ the code under the \textit{git} tag \textit{jackson-core-2.13.4}) because it is the latest stable version available at the time of writing. After verifying that the project meets the hard requirements related to GitHub (more than 2000 stars, more than 600 forks, 35 open issues\footnote{as of 2022-10-19 (ISO 8601 date)}), we ensured that the project had enough lines of code by using the cloc tool, which provided the following output shown in Figure \ref{fig:cloc}. By looking at the results we can finally assert that the project contains 58.787 lines of Java code and this satisfies all the requirements. \begin{figure}[H] \centering \begin{tabular}{lrrrr} \toprule Language & Files & Blank & Comment & Code \\ \midrule HTML & 4846 & 18473 & 235544 & 1997020\\ Java & 285 & 8532 & 20004 & 48783\\ CSS & 3 & 18 & 69 & 990\\ Logos & 2 & 260 & 212 & 605\\ Bourne Shell & 3 & 35 & 62 & 223\\ XML & 7 & 5 & 1 & 179\\ DOS Batch & 1 & 35 & 0 & 153\\ Markdown & 3 & 58 & 0 & 125\\ Maven & 1 & 13 & 23 & 112\\ YAML & 3 & 1 & 5 & 71\\ JavaScript & 1 & 1 & 0 & 29\\ JSON & 1 & 0 & 0 & 10\\ Properties & 2 & 0 & 16 & 5\\ \midrule Total & 5158 & 27431 & 255936 & 2048305\\ \bottomrule \end{tabular} \caption{Output of the \textit{cloc} tool for the Jackson Core project at revision \textit{jackson-core-3.13.4}.} \label{fig:cloc} \end{figure} \section{Analysis Implementation} The analysis is performed using the \textit{Pattern4J}\footnote{\url{https://users.encs.concordia.ca/~nikolaos/pattern\_detection.html}} developed at Concordia University. This program attempts to detect traditional design patterns by scanning the bytecode (i.e.\ the \texttt{.class} files) of a given project and by checking several heuristics. Due to the unceirtanty of this process we double-check each instance of a pattern found to use our own judgement and detect possible false positives. Since the tool needs compiled \textit{.class} files to perform the analysis, and since \textit{jackson-core} is a standard Maven project, we compile the sources using the command \texttt{mvn clean compile}. The \textit{pom.xml} of the library specifies Java 1.6 as a build target, which is not supported by JDK 17 or above. We used JDK 11 instead, as it is the most recent LTS version of the JDK to still support this target. An XML dump of the \textit{Pattern4J} analysis results is included in the submission as the file \textit{analysis.xml}. In the following sections each detection of the \textit{Pattern4J} tool is reviewed to characterize if it is indeed not a false positive and if the design pattern is varied in any way in its application. For the sake of brevity, when referring to a class by its fully-qualified domain name the prefix \textit{com.fasterxml.jackson.core} is omitted as all classes in the Jackson core project reside in this package or in a sub-package of this package. \section{\textit{Pattern4J} accuracy} As it would be very hard to check each class in the Jackson project for design patterns manually to get a true number of false positives and false negatives, we opt for a more statistical approach. We select 20 classes at random from the project and we review the reported results for this subset. The classes were selected from the \textit{target/classes} directory generated by Maven using the following command: \begin{verbatim} find . -name '*.class' | shuf -n 20 | sed 's#\.class##;s#/#.#' \end{verbatim} The selected classes and the analysis results for both \textit{Pattern4J} and our manual inspection are shown in Figure \ref{fig:conf}. Using those results, we can say that \textit{Pattern4J} shows a false positive rate\footnote{\url{https://en.wikipedia.org/wiki/False_positive_rate}} of 45.4\%, a false negative rate of 17.2\%, and an accuracy of 75.0\%. \begin{figure}[h] \begin{subfigure}{\linewidth} \resizebox{\textwidth}{!}{ \begin{tabular}{l|rrrr|p{7cm}} \toprule Class & \multicolumn{1}{p{1.5cm}}{True positives} & \multicolumn{1}{p{1.5cm}}{False positives} & \multicolumn{1}{p{1.6cm}}{False negatives} & \multicolumn{1}{p{1.6cm}|}{True\ \ \ negatives} & Notes\\ \midrule util.DefaultPrettyPrinter\$Indenter & ~ & 1 & ~ & ~ & state and adapter, false positives \\ ObjectCodec & ~ & ~ & ~ & 1 & ~ \\ type.WritableTypeId & ~ & ~ & ~ & 1 & ~ \\ util.DefaultPrettyPrinter\$NopIndenter & 1 & ~ & ~ & ~ & variation of singleton, true positive \\ json.PackageVersion & ~ & ~ & 1 & ~ & variation of singleton, false negative \\ io.UTF32Reader & ~ & ~ & ~ & 1 & ~ \\ io.NumberInput & ~ & ~ & ~ & 1 & ~ \\ json.JsonReadFeature & ~ & ~ & ~ & 1 & ~ \\ io.SerializedString & 1 & ~ & ~ & ~ & variation of singleton, true positive \\ type.TypeReference & ~ & ~ & ~ & 1 & ~ \\ JsonPointer & ~ & 1 & ~ & ~ & singleton, false positive \\ json.UTF8DataInputJsonParser & ~ & ~ & ~ & 1 & ~ \\ format.InputAccessor\$Std & ~ & ~ & ~ & 1 & ~ \\ JsonStreamContext & 1 & ~ & ~ & ~ & template method (barely), true positive \\ filter.TokenFilter & ~ & 0.5 & 0.5 & ~ & singleton, false positive; strategy, false negative \\ util.VersionUtil & ~ & ~ & ~ & 1 & ~ \\ json.UTF8JsonGenerator & ~ & ~ & ~ & 1 & ~ \\ JsonParser\$Feature & ~ & ~ & ~ & 1 & ~ \\ exc.StreamReadException & ~ & ~ & ~ & 1 & ~ \\ util.JsonGeneratorDelegate & ~ & ~ & 1 & ~ & decorator pattern (base class), false negative \\ \midrule Total & 3 & 2,5 & 2,5 & 12 & ~ \\ Percentage & 15,0\% & 12,5\% & 12,5\% & 60,0\% \\ \bottomrule \end{tabular} } \caption{Table of classes analyzed manually against the \textit{Pattern4J} tool results. Units are classes, the value $0.5$ represents a pattern inside a class with two patterns.} \label{fig:conf:tab} \end{subfigure} \begin{subfigure}{\linewidth} \centering \vspace{1cm} \begin{tikzpicture} \begin{axis}[ colormap={parula}{color=(white) rgb255=(90,96,191)}, xlabel=Pattern4J, xlabel style={yshift=-40pt}, ylabel=Manual detection, ylabel style={yshift=40pt}, xticklabels={Has pattern, No pattern}, xtick={0,...,1}, xtick style={draw=none}, yticklabels={Has pattern, No pattern}, ytick={0,...,1}, ytick style={draw=none}, enlargelimits=false, colorbar, xticklabel style={ rotate=45 }, nodes near coords={\pgfmathprintnumber\pgfplotspointmeta}, nodes near coords style={ yshift=-7pt }, ] \addplot[ matrix plot, mesh/cols=2, point meta=explicit,draw=gray ] table [meta=C] { x y C 0 0 3 1 0 2.5 0 1 2.5 1 1 12 }; \end{axis} \end{tikzpicture} \caption{Confusion matrix comparing the \textit{Pattern4J} detection results with our manual interpretation of the classes. Units are classes, the value $0.5$ represents a pattern inside a class with two patterns.} \label{fig:conf:mat} \end{subfigure} \caption{Results of the statistical analysis on the effectiveness of the \textit{Pattern4J} tool as a class-by-class table (Figure \ref{fig:conf:tab}) and as a confusion matrix (Figure \ref{fig:conf:mat}).} \label{fig:conf} \end{figure} \section{Structural Patterns} \subsection{Singleton Pattern} \textit{Pattern4J} found a lot of instances of the singleton pattern, namely 13. However, some discussion is required to understand the ways the Jackson core project applies this pattern, as the instances found are sometimes wildly different from the standard application or outright false positives. \begin{lstlisting}[caption=The \textit{sym.Name1} class.,language=java,label=lst:name1] public final class Name1 extends Name { private final static Name1 EMPTY = new Name1("", 0, 0); private final int q; Name1(String name, int hash, int quad) { super(name, hash); q = quad; } public static Name1 getEmptyName() { return EMPTY; } @Override public boolean equals(int quad) { return (quad == q); } @Override public boolean equals(int quad1, int quad2) { return (quad1 == q) && (quad2 == 0); } @Override public boolean equals(int q1, int q2, int q3) { return false; } @Override public boolean equals(int[] quads, int qlen) { return (qlen == 1 && quads[0] == q); } } \end{lstlisting} For example, \textit{sym.Name1} (whose sources are in Listing \ref{lst:name1}) has a package-private constructor and a \textit{public static final} instance of itself. This is enough for \textit{Pattern4J} to flag the class as a singleton, as its constructor is never called in Jackson core other than for initializing the aforementioned field. However, by reading the documentation it is clear that the class is meant to be instantiated multiple times. Indeed, its purpose is to box and represent JSON string literals shorter than 4 bytes, implying the class is meant to be initialized by clients of the core Jackson module. Several less-than-obvious results like this one are reported by the tool, namely: \begin{description} \item[sym.Name1, JsonLocation, DefaultIndenter, util.DefaultPrettyPrinter\$FixedSpaceIndenter] are not singletons and thus false positives. All these classes were detected because of ``default'' instances they include in themselves as \textit{static final} fields and because their constructor, even if \textit{public}, is never used in Jackson core itself. However, by checking the documentation all these classes are meant to be extended and instantiated in other Jackson modules; \item[JsonPointer, filter.TokenFilter] are as described above, however having \textit{protected} constructors; \item[JsonpCharacterEscapes, util.DefaultPrettyPrinter\$NopIndenter, Version] may be considered variations of the singleton pattern that however include a \textit{public} constructor that is never called in the module code, but that may be called in tests. Given the \textit{public} constructors, these classes are hardly solid singleton implementations. However, we gave the benefit of the doubt to Jackson developers as the use of the constructors in test code may hint to a purposefully open design to allow for testability; \item[io.JsonStringEncoder] is as described above, however the class is declared as \textit{final}; \item[util.InternCache, io.CharTypes\$AltEscapes] are both rather standard singleton pattern applications, however implemented with eager (non-lazy) initialization (i.e.\ storing the instance in a \textit{public static final} field); \item[io.ContentReference] is as described above, however having a \textit{protected} constructor instead of a \textit{private} one. \end{description} \subsection{Abstract Factory Pattern} \textit{Pattern4J} detects only two instances of the abstract factory pattern: \begin{description} \item[TokenStreamFactory] which indeed is a factory for \textbf{JsonParser} and \textbf{JsonGenerator} is a factory for JsonParser and JsonGenerator objects, although two overloaded factory methods exist on this class (one for each class) catering to a different combination of arguments. A concrete implementation of this factory is included in the form of the \textbf{JsonFactory} class, although other modules may add additional implementations to cater to different encodings (like the \textit{jackson-dataformat-xml} module for XML); \item[TSFBuilder] which is also a factory for concrete implementations of \textbf{TokenStreamFactory} allows slight changes in the serialization and deserialization rules (e.g., changing the quote character used in JSON keys from " to '). Like TokenStreamFactory, this class is only implemented by one class, JsonFactoryBuilder, within this module's scope. Moreover, as mentioned previously, this abstract factory will likely be extended by concrete implementations in other Jackson modules. \end{description} \subsection{Builder Pattern} The builder pattern does not seem to be analyzed by \textit{Pattern4J}, as the analysis output does not mention the pattern, even to report that no instances of it have been found (as is the case with other patterns, e.g., the observer pattern). A manual search in the source code produced the following results: \begin{description} \item[TSFBuilder] is also a builder other than an abstract factory. As mentioned previously, this class allows slightly altering the serialization and deserialization rules used to build outputting JsonFactory objects. Each rule is represented by an object or enum instance implementing the util.JacksonFeature interface. TSFBuilder then provides several overloaded methods to enable and disable features represented by the interface. Enabled features are stored in several bitmask-protected int fields, which are then directly accessed by the constructor of the TokenStreamFactory concrete implementation to build; \marginpar[right text]{\color{white}\url{https://youtu.be/72b2nH-kdbU}} \item[JsonFactoryBuilder] is a concrete factory implementation of \textbf{TSFBuilder} that builds \textbf{JsonFactory} instances; \item[util.ByteArrayBuilder] provides facilities to build \textit{byte[]} objects of varying length, akin to \textbf{StringBuilder} building \textbf{String} objects. This is not a strict implementation of the builder pattern per se (as Java arrays do not have a ``real'' constructor), but it is nevertheless included since the features it exposes (namely dynamic sizing while building) are decoupled by the underlying (fixed-size) array representation. \end{description} \section{Creational Patterns} \subsection{Adapter Pattern} \textit{Pattern4J} found many instances of the adapter pattern, however all but one wVere shown to be false positives by checking the documentation and the code using each allegied adaptee. The matches found are reported and commented below. Matches are shown in the \textit{[Adapter] $\leftarrow$ [Adaptee]} format. \begin{description} \item[JsonFactory $\leftarrow \{$ sym.ByteQuadsCanonicalizer, io.InputDecorator $\}$] false positives, by reading the documentation it is clear the classes have different purposes and \textbf{JsonFactory} is merely using the other classes' functionality through composition; \item[base.ParserBase $\leftarrow$ json.JsonReadContext] false positive, \textbf{json.JsonReadContext} is instantiated several times in \textbf{base.ParserBase} by mutating the container field with the new instances; \item[base.ParserBase $\leftarrow$ json.JsonWriteContext] false positive, like above; \item[$\{$ util.DefaultPrettyPrinter, util.MinimalPrettyPrinter $\} \leftarrow$ util.Separators] false positives, another example of instances used through composition; \item[io.SerializedString $\leftarrow$ io.JsonStringEncoder] indeed an adapter pattern application, although the adaptee backing field is \textit{private static final}. \textbf{io.SerializedString} is a class that wraps a String and allows it to be encoded using the \textbf{io.JsonStringEncoder} static instance, storing the result and re-using it in case of multiple serialization request (a tecnique similar to memoization). Therefore, the main purpose of this adapter is not to adapt against any interface, but to wrap the functionality of the adaptee and store its results for re-use; \item[util.DefaultPrettyPrinter $\leftarrow$ util.DefaultPrettyPrinter\$Indenter (2 fields)] false positives, both enclosed fields are simply used in a composition relationship. \end{description} \subsection{TBD Decorator Pattern} Decorator pattern lets you dynamically change the behavior of an object at run time by wrapping them in an object of a decorator class. \begin{description} \item[JsonGenerator] TBD \item[JsonParser] TBD \end{description} Only in Pattern4J \subsection{Bridge Pattern} TBD \section{Behavioral Patterns} \subsection{State Pattern} Among the design patterns \textit{Pattern4J} detects, the state pattern is detected in 5 classes. The state pattern is a variation of the strategy pattern where the concrete strategy used by the matching context is determined by the state of a finite state machine the context class implements. In other words, the state pattern chooses the concrete strategy to use through the state of the context. By analyzing the \textit{Pattern4J} results and the code, we can say that all the instances of the state pattern the tool finds are false positives. Namely: \begin{description} \item[util.DefaultPrettyPrinter \textit{\_inputDecorator} and \textit{\_outputDecorator}] are fields flagged as states, thus flagging the class as a state pattern instance. However, no ``state'' akin to a finite-state machine is maintained by the class to determine which implementation of these fields to invoke. What is detected are more likely lightweight implementations of the strategy pattern, since these fields can be mutated through matching getters and setters. Additionally, the documentation of each of the matching \textit{...Decorator} field types (namely interfaces) states that implementors are meant to be algorithms to pre-process input before the formatting process (a feature labeled as ``decorator'' w.r.t.\ the library, not to be confused with the decorator pattern); \item[util.DefaultPrettyPrinter \_objectIndenter and \_arrayIndenter] are false positives too, and are likely strategy patterns too for the reasons described above. \item[util.DefaultPrettyPrinter \_rootValueSeparator] is flagged as a state field too, however the field is simply a boxed \textit{String-like} immutable data structure (i.e.\ \textit{SerializableString}) that is swapped during the pretty-printer parsing logic; \item[json.WriterBasedJsonGenerator \_currentEscape] is a false positive for the same reasons described above. \end{description} \subsection{Strategy Pattern} \textit{Pattern4J} detects no instance of the strategy pattern in Jackson, however the previous section regarding the state pattern referenced some false positives that were indeed applications of this pattern. Due to the flexibility of Jackson, there are many more instances of the strategy pattern to configure and customize the serialization and deserialization pipeline in several stages. \subsection{Template Method Pattern} Due to the extendibility of Jackson, it is of no surprise that the template method pattern is used extensively to create a class hierarchy that provides rich interfaces while maintaining behavioural flexibility. \textit{Pattern4J} correctly detects several instances of the pattern, namely \textbf{JsonStreamContext}, \textbf{JsonGenerator}, \textbf{type.ResolvedType}, \textbf{JsonParser}, \textbf{base.ParserBase}, \textbf{base.GeneratorBase}, \textbf{base.ParserMinimalBase}. All these classes implement several concrete \textit{public} methods throwgh the use of \textit{protected abstract} methods. Although the concrete (i.e.\ the template) methods are usually not vary complex (as the pattern example shown in class), they still follow the principles of the template method pattern. We show as an example some template methods found in \textbf{base.ParserBase}: \begin{lstlisting}[caption=Template method \textit{void close()} and step methods \textit{void \_closeInput()} and \textit{void \_releaseBuffers()} in \textbf{base.ParserBase}., language=java] @Override public void close() throws IOException { if (!_closed) { // 19-Jan-2018, tatu: as per [core#440] need to ensure no more data // assumed available _inputPtr = Math.max(_inputPtr, _inputEnd); _closed = true; try { _closeInput(); } finally { // as per [JACKSON-324], do in finally block // Also, internal buffer(s) can now be released as well _releaseBuffers(); } } } protected abstract void _closeInput() throws IOException; protected void _releaseBuffers() throws IOException { /* implementation omitted */ } \end{lstlisting} Here the pattern is slightly modified by providing a default implementation of \textit{void \_releaseBuffers()}. In this case, child classes occasionally override the method with a body first calling \textit{super()} and then adding additional buffer release code after. \end{document}